Perhaps you are blocking your mobile phone's outbound caller ID by default? Are you calling directly from your mobile phone's native carrier number, and NOT your Google Voice number via the Google Voice app?
I don't use the AA, but I recall experimenting with it, and running into your issue - it unfortunately doesn't have a "do what I meant" feature, and it expects to see the exact inbound caller ID.