For me it's always been a very distinct line--cops good, rent-a-thugs very bad.
I shoot at night, so I certainly get my share of attention. but have never had a real cop spend more than 10 minutes asking reasonable questions (although sometimes including leading questions trying to get me to admit to commercial intent, which would require a permit).
But I've had rent-a-thugs tell me I needed to stop right now or I was going to jail. (I didn't stop, I didn't go to jail, he didn't even call the cops), I've had them trot out all sorts of copyright, trademark and terrorist malarkey, I've been threatened that they were "going to call the cops if you don't leave right now" several times, and never ONCE did they call when I told them I wasn't stopping. (Note that I was never on their property.)
All this despite the fact that I generally AM doing something illegal! San Diego, like many cities, has gone to great lengths to criminalize homelessness. As part of this, it's an infraction to set any personal property on a public sidewalk for purposes other than the immediate loading/unloading of a vehicle. This would appear to mean I could and should be cited for having my tripod on a sidewalk. But it's never happened, even when it was the real cops stopping by to check me out.