I would keep all of them if you have room. I have a Beseler 23C which I use for enlarging and an Omega C760 which I use only for making contact prints. Since I have to fit in my darkroom time around all of my other obligations I find it very convenient to have one enlarger already set up to for contact prints. Since I use a slot processor for everything 8 1/2 x 11" and smaller, I can literally walk into my darkroom and make a few contact sheets in 5 or 10 minutes.
If I had a third enlarger, I would leave it configured for medium format.