I have similar config too, GV+Callcentric, however, I've been mainly using GV for almost all my calls, I am using a airlink AR430W DD-WRT as router.
I initially configure no QoS and protocol based (SIP, RTP) QoS on my router, this did not help, I then changed it to MAC based and IP base QoS, and saturate both my downlink and up link DSL channel with P2P upload/download and remote desktop updates, GV quality remain un-affected
My guess is that protocol based QoS takes a lot of CPU on the router to do deep packet inspection and does not work well, MAC/IP based QoS is better for low end router