I shoot 6x9, 6x7 and 6x6, and all three formats are fun. But the format that really caught me in the first place, and in my mind really reveals the biggest quality leap from 35mm, is 6x4.5. It isn't the sexiest of the medium formats, and there are some who would like to make you believe that they are almost not worth the trouble, but those 6x4.5 images are magnificent. In reality the larger formats can sometimes be disappointing because they don't give that same sudden quality jolt that 6x4.5 can give. And fortunately that format is also very well served by some terrific cameras and is usually the least expensive to start with.
For a long time my next favorite medium format size was 6x9. To be honest, I am a folder freak and have dozens of them lying around, some working, others in various states of disrepair and repair. There are days where I have spent many enjoyable hours hiking through the mountains in my area with a 6x9 folder at hand. But I have to be completely honest, lately the 6x9 folders have been staying home and I have been carrying my newly purchased Fuji GF670. It may be a bit pricey, but I can't think of a lighter, easier to handle, camera for hiking that has the ability to produce the same high quality images. A late model Rolleiflex may provide the same quality images, I don't know, but the price of admission to that club is wayy higher.
All the medium formats are fun, but if I could give you one very strong piece of advice, it would be to buy the best camera you can afford to own, regardless of the format size. In addition, if you intend to move away from 35mm completely, then be sure to get one that allows you to exchange lenses, even if you start with only one. Finally, buy one that can still be repaired. Face it, we all try to buy the most reliable cameras we can find, but s..t happens.
Other than that, have fun!!