All of Google's services are associated with a Google
account. Maps, Gmail, Docs, Calendar, Hangouts and Voice all share one user ID/password.
If you created a Google/Gmail account for GV #1, and you still know the user ID and password, and you can sign and see the inbound Google Voice number here:
https://www.google.com/voice#phones , then that number is linked to that account.
Similarly, you would need to find the user ID and password associated with the other Google Voice number. There is no "somehow"; there is an account. If you don't have any record of the account, then you have lost the number.
If you have written down various email account user IDs and passwords, you can use the tool at the page linked below to have Google send an automated email note to the correct email account that holds the number. It will be entirely up to you to have kept track of the account credentials. When you first set up a Gmail account, Google asks you to also add a recovery email address (a different email address to send a password reset in case you can't get into the primary account). It also asks you to supply a recovery telephone number for the same purpose. If you did neither, then there is no hope of recovering the number.