"In many cases" you say? In what cases would you need port forwarding then?
If you have a cheap or otherwise low quality router that doesn't perform well with VoIP, port forwarding may be necessary.
Forwarding the SIP port may solve the problem of your phone failing to ring.
Forwarding the RTP ports may solve the problem of no inbound audio, or slow audio setup.
Forwarding ports will NOT impact poor audio quality.
However, drgeoff is correct. Port forwarding should be done as a last resort to solve a specific problem - as it is a security risk.