A more suitable reply to the upset woman would've been: "Oh I'm truly sorry if you think I'm some kind of pervert, I'm a photographer and I'm doing a documentation about the people on the streets in this town, here take my card, drop me an email or call me and I'll send you a cpl of prints for free."

Normally this is what I do if someone objects after I've captured them (rare though...) and so far no unhappy faces after that, instead nice conversations about the town, about photography, bla bla etc.

Good luck m8!!