"Why" is always an excellent question. For me it was big prints, and I can get flawless 16x20s from 6x7 even with slightly fast film (TMY2) and you can push out to 30" with some care and a fine (Acros/TMX) film. The quality you get from 35mm on 8x10, you will now get at 16x20.
Yes, 6x7 Mamiyas are large. However I haven't found that to be a problem with travelling as I can still fit the body, three lenses and a bunch of film into a backpack and carry it around all day (but I'm a large 30ish guy, you might decide differently if you're smaller or have a chronic injury). I also find that the RZ (and by extension, RB) is eminently hand-holdable if you keep the shutter speed up to 1/F. In particular with the WLF you can jam the back of the camera against your chest, support the end of the lens and have excellent stability. Or use a long/heavy lens (250 APO) with the prism (heavy++) and the moment of inertia of the camera is suddenly so large that the mirror slap no longer rotates the camera much and you can get pin-sharp handheld candid portraits if you've got enough light and/or film speed. The only hard part is nailing the focus at f/4.5 or f/5.6 as it's about one eyeball deep.
If you want smaller/lighter yet with a bigger negative, there are a bazillion 645 and 6x6 choices. I don't think 645 is a meaningful improvement over 35mm, but it's definitely a start and the ergonomics can be as good as 35mm. 6x6 is getting there unless you crop to a narrow aspect ratio, in which case it's no better than 645. All of those cameras are a lot smaller+lighter than a Mamiya 67.
If you have cash to burn, look at a Mamiya 7. Huge neg, unbeatable lenses and a tiny package... for which you will pay dearly.
While I do 4x5 and enjoy it immensely, I don't think it's in any way a replacement for a MF SLR, especially if you travel. While a 4x5 field camera and 3 lenses is much lighter than an RZ67 and 3 lenses,consider the bulk of 5 film holders (roughly 5x7x3") vs a single roll of 120 (1x1x2.5"), and how much time you're going to spend loading and unloading those holders while risking dust... all for a neg that's only half as much advance on 6x7 as 6x7 is from 35mm.
Perhaps you could hire a couple of different cameras for a weekend to try them out? I suspect though that you could easily blow your whole camera budget that way: end up knowing exactly what you want and have spent the $200 or whatever.