Well I did just that in the late 1980's and bought an M3 Leica. However I had by moved to K mount cameras a few years befor and my M series Pentax cameras were very much quieter than my Spotmatics.

I didn't change becasuse of the shutter noise but it's a valid observation which people ask about.