He also has the fredom of speach to ask you to stop (nothing wrong with that).
IF you are too close and continue to photograph in a harassing way he might choose to get the police involved, public place has nothing to do with it. If he is a private person once he asks you to move on you should, no need to argue anout public places and such.

I am a professional with some 25 years of experiance and few photos are worth upsetting the public about, or spending a night or two in jail. There are lines I WOULD cross as a professional and would be happy to spend a day in jail right or wrong to provide an employer with an image. The world HAS changed, the law is flexible, just becuase you think case law is one your side, upsetting a private person with continued picture taking no mater how cool they look is just itching for a fight. Now if they were involved in a protest line etc fine.. if they are sitting against a tree alone, take the shot or two and move on...