I use a voigtlander R3M + 21mm f/4

I've been using this lens for about a year. Note that I usually like to supplement this lens with the 40mm f/1.4
The viewfinder that is supplied with this lens does not have the screwy thing (like on most flashguns) to lock in in place.

There has been one instance where the viewfinder has slipped out.
Now I'm afraid to leave my camera on my shoulder in case the viewfinder drops while I'm looking at oncoming traffic. (That is one example instance when I should be looking at something else and not my camera)
Do you have any solutions?
I also like to use off flash trigger on my rangefinder. So the attachment can't be permanent.