Format only "matters" if it gets in the way of getting the image you want. I agree that Vivian Mayer's El train work would have been impossible with an 8x10 (even with a Hobo), but 4x5 would not have prevented her from shooting that work- look at Weegee's reportage, or sporting event coverage from the first 65 years of the 20th century - there's a famous picture of I believe an early Muhammad Ali fight, and almost every photographer ringside is holding a Speed or Crown Graphic.
Bill- as regards your street shooting with the 4x5, that's just a matter of practice to get it right. A few hundred more frames and it will become second nature
I've been going back through my old negatives recently, and scanning a bunch of stuff to either add to an online portfolio or send images to folks in email. I keep being pleasantly surprised by the small-format stuff I find, so I'm not renouncing that work by any means - if nothing else, I'm rediscovering how good it can be, and it's drawing me back into using it more. I'll be taking my Rolleiflex with me to Cuba in March - I want the quality of the bigger negative, but with the shooting speed of something hand-held.