Couple more thoughts on this problem, and I assume it is the condensor variety.
The condenser setting needs to be changed for the different negative sizes. The condenser head needs to be move up from the negative stage when using 35mm and moved closer to the negative stage when using larger formats. You should have an adjustment bar on the right hand side of the enlarger.
Next are you using the same lens for all your formats. A lens that covers 35mm will not give good coverage with a 6x6 negative, you should ideally be using something like this:
35mm negative = 50mm enlarging lend
6x4.5mm negative = 75mm - 80mm enlarging lens
6x6cm = 80mm enlarging lens
You could of course use an 80mm enlarging lens for all formats but you will be limited wth the magnification you can get when you use an 80mm lens with a 35mm negative.
I would check these out first before purchasing a new lens.