Street photography, for some people, seems to be like getting a needle...the anticipation is worse than the actual thing itself.
Just do it. If someone objects enough to say something, be simple and straightforward. I've been doing this in my downtown for over 2 years now, and haven't had an objection. I've struck up a few conversations, but never a problem.
Be at ease, know what you're doing and more importantly why.
Most times I just wander and shoot if something catches my eye. I pre-focus and have my settings right beforehand. I'm stealthy but I'm not dishonest. I'm part of the passing scenery just like everybody else is. Beyond that, I don't know...you gotta just go out and do it.