The Pentax 645N and 35mm combo is the only reason I am shooting MF in my mix of formats and cameras. The lens is spectacular and a bargain (even though they're now selling for nearly twice what I picked mine up for some years back.) The camera is not physically larger than a large 35mm SLR like an F5. The viewfinder is better than a 35mm SLR (yes, even the F5). It is very handholdable down to very slow shutter speeds as it has a well-damped mirror, at least it is for me (I've gotten tack sharp results to 1/8s, handheld).
Can't speak to the Fuji, but two things an SLR has over a RF is no focusing parallax error, and no parallax framing differences.