Hi, sorry I wasn't around earlier to post:
Google is aware of this issue, impacting XMPP-based Google Voice calls. As you've described, the symptom is typically one party not hearing the other.
They have identified the likely root cause and are working on a fix. No estimated time to recover is available, but rest assured that they're on it.
As they used to say on TV, "Do not attempt to adjust your set." It has nothing to do with your device or network connection.
For issues like this in the future, please feel free to check for similar posts, and/or report your symptoms over on the Google Voice Help Forum (linked in my signature block).