I have a different theory... the only time I worry about dust is just prior to putting a neg in the carrier! Regardless of how careful and clean you are you still need to clean the neg before making a print.

I do vacuum the floor occasionly (with vacuum system that's located in the garage and runs thru the house walls.. can't think of what their called at the moment... so it doesn't expell dust filled air back into the same room!) and wipe the benches every now and then. When I get the urge for a bigger clean up and move the enlarger, paper safe and bits 'n 'pieces lying around, they usually have heaps of dust under them.