If someone takes your photo in the street and you punch them for the intrusion, it's not the photographer that's getting arrested.

I'm not as active as some people here, and I don't have the benefit of being in a big city (Raleigh truly is a small town), but I've never had more than a couple of dirty looks cast in my direction. The police are the worst. If they see a camera aiming at them, that is.