Also... Don't forget the ceiling!
I have white walls and white ceiling. When I burn in my prints, I can tell areas that was not supposed to gain any exposure does. My enlarger, Omega D2 leaks light horizontally and UP. Obviously, reflected lights are reaching my paper on baseboard.
I've seen some darkroom practioners create a black surround for enlarger only. That may be an option too.