I assure everyone here that I: 1) I am not some badass photographer just itchin' for a fight; 2) the woman was NOWHERE NEAR the angle of my lens or in any part of the composition that would lead any reasonable person to believe they would possibly be in the picture; my camera was pointed towards the top of the Flatiron Building, which is at least 25 stories high; and 3) I tried to very politely tell her that she was mistaken about being in the line of fire of my shooting. I only told her that she should stay home when it was VERY clear that she was trying to pick a fight with me and that she had a right to privacy and was sick and tired of photographers trying to take pictures of her.
Upon further reflection after reading some of the responses here, I do believe that she was truly a sad, borderline schizophrenic person who I somehow feel had this conversation numerous times before she had it with me. Especially after she told me she that she was fed up with photographers.
As to people's desire for privacy in public - of course I can respect that. But this, alas, does not equate to a right to privacy. I wasn't trying to take a picture of her. But even if she was in the picture, and not the subject of it, my right to photograph trumps this is NY, especially because I am not using the picture for trade or advertising reasons.
As to the exterior of buildings being protected by copyright in parts of Canada - really? Wow.