Well I happen to also have, and shoot with, a MF SLR, in my case a Mamiya 645 Pro with the AE prism and winder grip.
Bottom line is that the 645 is more versatile, of course, but I actually enjoy walking around and shooting with the Yashicamat more. It's unobtrusive, gets lots of smiles and favorable comments, is very quiet, and unlike the 645 Pro I never feel like I have a cinder block, or an albatross, around my neck.
I have shot several 35mm cameras, a lot of medium format cameras, a few large format cameras and a couple digital cameras. What I have found is that you need to shoot whatever makes you happy. That's all that matters.