White is advisable as with less means you gain an even illumination.

Comcerning reflectect exposure light bouncing back to the paper, a shielding of the paper can be applied. The most simple form would be a black painted section behind the enlarger. Alternative one could use a black roller blind. Also mobile shieldings at other sides of the enlarger. It's a question of weighing between the "trouble" of shielding versus the lighting efficiency gained by white walls.
Local black shieldings could also be used in provisional darkroom. Think of black card/foamboard too.

But what about the white shirt of the Apugger?? Thought about that?

why not look for a wall-paint without optical brighteners? Are the paint buckets all sealed today? Otherwise make a check with a UV-lamp before buying.