If you didn't blow the dust off neg between prints, then the specks or hair were probably on the paper when you had the enlarger light on. Doesn't matter in the developer after that, you can scatter all the dust you want at that point. Sometimes there are specks caused by stop bath being so strong they cause effervescence but doesn't sound like that to me in your case.
I for one, would rather have a white speck to spot on the print... than a black spot (caused by dust on the film before the light hits it in the camera). That's harder to retouch.