Those 4 screws should release the diffusion filter and frame assembly, and the rods should remain together within the frame. Any cleaner you mentioned should work, or just soap it up good with dishwashing soap and rinse. Then blow it out and let it dry.
Originally Posted by frotog
I would not worry about the dichro filter. Remember, the light goes though the diffuser, and then through the plastic diffusers in the mixing box. A few blemishes shouldn’t hurt.
But I would clean all the dichro filters, and the IR and UV filters, before I reassembled the enlarger head. I cleaned all my filters by removing the glass from the frame, and soaking it in warm water with lots of dishwashing liquid. Rinse it off in warm water and blot with a paper towel. By being warm, the glass dries fast.