Try using these with the aid of q-tip's for applicators ( disposable cotton buds wound on the end of a short piece of small dowel if q-tip is a new trade name to you)

Start with mildly dilute household bleach, to kill the mold.
Then a cool water rinse.
Then a try with hot water with some household dishwashing detergent added.

Let us know how that works.

I would start on the outside, away from the lens as an area to test your technique.