No, I wouldn't punch him. I'm not inclined to violence anyway but to me the idea of punching someone out because they were rude or inappropriate towards me would actually be a worse assault than the one they committed. Having said that, my opinion of him as a person and as a photographer is pretty low based on what I've seen so why should I care?
Anyway, it's America. Someone is bound to shoot him before long.