If you are on private property such as a mall and someone (security guard) tells you to stop taking pictures comply. I'm not a lawyer but I've never heard of someone on the actual street snapping away without harassing people having a problem. Just tell anyone that asks, you are a tourist from Canada taking pictures of American culture and sites. If you stop when people ask you to stop you shouldn't have a problem.
If you are traveling with 35mm film the US guys should be okay. Medium format film is where things can get weird since I guess they don't see it often in a lot of places. They have always swabbed it in my experience but some dolts want to take it out of it's sealed wrapper. If I am going to be going through a check point with a boat load of medium format film I email a week or so in advance and give the manager a heads up that way they can make sure the staff complies with the proper swabbing procedure.