Here's a video from Google explaining how this feature works:
The My Location feature is available on a few selected devices right now.
Regarding the all important question that lot of people would have, this is what Google Help says:
Will Google always know where I am if I use My Location (beta)?
No, Google does not know who you are when you use Google Maps for mobile. All handsets are anonymous. When you use our product, we do not collect personal information like your username or phone number, so we do not know who owns or is using the handset. Therefore, when we identify approximate location [as part of the My Location feature], all we can determine is the approximate location of the handset. This location information is also only in our system when a user has opted-in to the My Location (beta) service. If the My Location (beta) feature is disabled, we will not continue to send radio information back to Google servers to determine your handset's approximate location while you use Google Maps for mobile.
Good job answering that question upfront!
Android Update: This feature is sure to be available in future versions of Android (if it is not already). This means, any Android application which needs your location information would work, kind of, even if your Android cell phone does not have GPS! Wow!
IMPORTANT "My Location" Update: Google Maps mobile with "My Location": Privacy Loophole
[via the very new Official Google Mobile Blog and Bits blog]