With 35mm, I don't get much comments. I use old cheap rangefinders and am quick about taking the photos, so most people don't notice. The only ones who seem to say anything are old Navy Vets who used to use the same gear when they where in the service.
4x5 is a different beast though. Most people just ask "Can you still get film for that thing?" (Course not! I'm just setting this big camera up to look cool) Using a Speed Graphic though, people usually just assume I'm doing what I'm doing for a very important reason and just stand a ways off and gawk. Those who do ask why I'm shooting whatever it is I'm shooting, I just tell them its for the town's historical record.
A 4x5 or larger seems to be able to wield a lot of control over bystanders as well. Many uses.