I think it matters with true condenser enlargers, using a point source. But these are very rare. What most of us call condenser enlargers are in reality condenser/ diffuser. They have ground glass in them, a large opal bulb and no bulb adjustment - in which case I think it doesn't matter much.
I have both 6 X 6 and 35 mm enlargers from the same company, so they share a common lens mount. I often use the 35mm lens (50mm focal length) for 35 mm negs on the 6 X 6 enlarger because it is easy to swap lenses than swap enlargers (my bench isn't big enough for both). I reckon if anything the 6 X 6 is slightly superior, as it has better condensers (two opposing lenses instead of one)