Thanks for the tips. I was going to try the MaxSessions route, but then I decided to take another look at the IP phone settings. There was no call waiting on/off option that I could find, but there was an option to set the call waiting tone. So I figured out how to make it silent, which was easier said than done, because just clearing the field resulted in a default beep.
So now if I'm in a conference, and someone calls, the phone gives a visible indication with caller id, but there is no noise added to the current call.