I have only had three people approach me while photographing Downtown in my area.
The first one was, to put it nicely, not the sharpest stick in the pile. I answered his questions politely and he moved on.
The second one was an employee of one of the buildings that I was photographing. The building happened to be a restaurant and he thought that I was from the state health department gathering evidence to close them down. I explained to him that my hobby was photography and I was not out to close down the restaurant. Simply taking black & white "art" style photographs.
The third person was a cleaning lady at one of the buildings. She was coming to work while I was photographing. When see got out of her car she saw me standing there and noticed the where she parked would be in my way. She got back in her car and move it out of the way. After finding another parking spot she came over to me and started talking. She made a comment about my beautiful camera (Zone VI 4x5) and asked a few other questions and finished by saying that she would love to see the photographs when finished. I told her that I would be happy to send her a print, so she gave me her bosses business card and ask me to send it there.
I have found that most people just ignore me while photographing. But then again I live in Des Moines, Iowa