It sounds like that woman has some kind of emotional problem or maybe she's just perma-paranoid because she's over/under medicated.
I've had adverse reactions when shooting street. It's part of what makes the form so difficult to do well and it's a big part of the reason not everybody does it. But that woman clearly went above and beyond to create turbulence where none was needed nor called for. Your actions as you described in no way warranted being called a pervert.
Cheer up! Don't let some stranger's toxic words poison your art. I'm sure this is much easier said than done!
Anyway, really sorry to hear about it.