Sounds a little like my Zeiss Ikon, as it went away after a couple of turns of the focus ring, I never bothered to fix it, in fact I think it went away by itself.

However, if it bothers you, sounds like a simple service would fix it. Maybe talk to CameraQuest, they seem to be the go-to place for Cosina/Voigtlander.