But I always enter the # sign after entering the digits. So the number should get processed immediately without any delay.
That's true.
I have three VGs setup on an OBi110 and there is no noticeable delay in calls being set up using any of them. It may be a problem with the service provider you are using on the VG.
Most providers will allow you to make calls without registration, but not receive calls. As an experiment it might be worth setting up a VG with one of your "faster" service providers on sp1 or sp2 and compare results.
A very rough guide to see if the call has been processed by the OBi110 is to click on "Call Status" on the local web page of the OBi110 every second or so after you finish dialling. It will show blank until the call has been processed and sent out of the spX that the VG is using.