I don't wear gloves but wash/dry my hands before handling film carefully from the edges.
I use an antistatic brush first and then an air blower, both after placing the film in the carrier if it happens to be glassless.
Almost never got any dust printed (it actually amazes me).
I also keep the brush clean and in its box when not used.
If your gloves are usual cotton ones they may well be the source of fibres. I suggest using polyester antistatic gloves if you have to use them at all.
This and air blower is where I would start.