You haven't provided enough detail about your network setup, location, and anything else you can think of, to really dive into this -- it would just be total SWAG guesswork at this point.
I doubt it's a Google Voice bug, but you could try creating a different Google Voice account, obtain a different number, for testing purposes, provision it on your two devices, and make/receive calls for a while. This would at least narrow the possibilities.