A nut-job in Central Park dressed up like Bootsy Collins on way too much acid gave me a verbal dressing down for trying to take his picture, after I threw 5 dollars in his hat for the 'performance' he was putting on.
A saxophonist in Berlin did the same thing. I took the money I gave him first out of the hat and told him to FO2.
I don't mind, just don't give it if you can't take it. Now I carry little business cards and if I take someone's picture, I give them a card and tell them I'll send them a copy. That goes a long way :-)