Personally I don't see a need to get a bigger negative (and camera) if you print 16x20 or maybe even 20x24. If you stand more than three feet away from the print, you're not likely to spot much difference. It depends a little on what film you use.
A properly exposed and processed negative in 645 format will yield a virtually grain free 16x20s. Heck, shooting Acros/Xtol in 35mm I get almost grain free 16x20s; you'd have to press your eye up against the print surface to see it.
My advice - In interest of the pictures themselves, you're best off shooting the camera that feels the most natural in your hands and fits into your work flow best. Unless you like to look at your prints with a loupe. Those are my two cents.
I agree with Thomas. 30x40 is about the limit I go with 645 so a 20x24 is no sweat for this format. I use a Mamiya 645E and use it when 4x5 or 8x10 is impractical.