OP...35 years ago in a park in L.A. I was hassled something like that. Kids are great subjects to shoot. But it is a problem in this day and age. Glad you stood your ground. as long as we are legal, nothing to worry about. (although there is always the crazy person and odd circumstance we may deal with.)
Good luck and keep on blasting away.
...and yes we are always looking for the next shot!