I have near-white walls. I also have white shelves, as it is a small space and there would be a lot of pockets of shadow without the reflection otherwise. I have a 'skirt' of blackout fabric around the carrier on one of my enlargers because it seems more prone to lateral light leakage and is closer to the wall than the other. On balance I think a neutral (white, or grey, or black) is better than a colour. Safelights can vary from red through orange to light brown, or green in a colour darkroom (though with my eyesight a dark green light is next to useless 8-) ). A neutral should be equally effective in all cases.
A good washable surface is a good idea.