I wonder how many people who disregard the M5 have used one on a regular basis, or are just quoting what they've read...
I have carried an M5 (and an M3 and a CLE) all over Afghanistan and the M5 is still - as it was prior to that - my favorite M camera for real usage. It doesn't look as sharp as the M3 or the other M's, but it consistently works miracles - with the emphasis being that it WORKS. I am 7,000 miles from my camera repair guy and I can use the M5 out here with confidence knowing that it will consistently function and yield incredible pictures every time. I love the CLE - mentioned by someone earlier - but it's not built to the same tough standards as the M5 and if it breaks there are NO PARTS for a repairman to use to fix it. The M5 is larger than the other M's, but not too large. Mine is slightly bigger than my palm and I carry it with one hand, only raising the other to focus - same as with the smaller RF's. WIth a 28mm, f/2.8 it's still an unobtrusive camera. And it looks WAY COOL with a DR Summicron attached.
If the camera works as well and reliably as the other M's, then it boils down to a matter of preference. I prefer the M5.
It took me a roll or two to get reacquainted with spot metering, but now I remember why I liked it so much as a young photographer. You can have the meter calibrated for modern batteries if you prefer, but the mercuries are still available overseas.
I recommend the three lug version which has a 28mm frame. That's what I use most often.
As mentioned before, people either love or hate the M5. If you're happy with what you've got - don't change. If you don't care for your current setup, try an M5 - borrow one if you can and shoot with it for a week. You'll know the first day if it's right for you.