There's plenty of people-watching in the Castro, and generally they ignore you if you look and act like a tourist (ie don't get intrusive into their personal space). I would NOT go wandering around the Tenderloin or Polk Street with kids in tow, but there's plenty of characters around there (be careful who you point your camera at, as the hookers, johns and pimps may not take too kindly to what they could mistake for police surveilance). The Mission district is a great spot for people-watching - along Valencia between say 15th and 20th (and you'll find a killer meal while you're in the neighborhood!). Take a drive out Geary Boulevard and pass through the Korean, Vietnamese, and Russian neighborhoods. For a quick break from the urban scenery, at the very end of Geary Boulevard, just before it drops downhill and to the left at Cliff House, turn into the parking lot on your right, above the ruins of the Sutro Baths. You can descend to the baths ruins, and walk on the beach to take in Seal Rock, then climb back up and follow the cliff-top trail along the coastal headlands of Lands End. It's a beautiful patch of total wilderness with the sea breezes, salt air, and the crying of the gulls, a few hundred yards from total urbanization. Go about a mile along the trail and you'll have amazing views of the Golden Gate Bridge.
Another terrific nature break from the city are the Open Space Preserves, halfway down the peninsula, between Half Moon Bay on the coast and Palo Alto on the bay side. Purisima Creek OSP is relatively easy to get to (head south on highway 1 through Half Moon Bay, turn left on Miramontes Road, right on Higgins Purisima Road, and enjoy the scenery until you get to the park entrance (there's very little parking at the entrance, so you'll probably have to park on Higgins Purisima Road or Purisima Creek Road and walk back)). You'll be walking through a grove of giant sequoias along a canyon carved by a stream with several waterfalls along its course, and there will always be banana slugs to entertain the kids. On the way back to SF, swing through Palo Alto, get something to eat on University Avenue, take a drive down El Camino Real to California Street and poke your head in Keeble & Shuchat to do a little camera shopping.