I'm very surprised that still nobody posted some reason why the idea wouldn't work, it seems very convenient and straightforward and I wonder why nobody thought about it before. I mean: first enlargers were simply view cameras put back to back with a window, using daylight as light source. I wonder why the same technique shouldn't work now with 35mm.
I strongly encourage you to make an experiment with the enlarger under sunlight. Unfortunately I only own enlargers with mirror-reflected light source and can't make the experiment myself, but I am extremely interested in reading your results.