Get on the Bus!
Every time I move to a new city I make sure to jump some form of public transportation. I have a car, but there's something about getting on a bus or metro that allows you to see a new city from an insider's perspective. Most of the time you can just hop right back on and go back the same way you came. So far I haven't gotten lost.