How rough is "rough"---are you going to go around driving nails with the camera? The Voigtlaender rangefinders have a reputation for being somewhat prone to getting knocked out of alignment---I haven't found it to be a problem with mine, but I'm not all that rough on cameras. Maybe the folks with more recent Bessae can say whether the alignment has gotten more solid since the Bessa-R.
The Soviet rangefinders, of course, are inexpensive and nigh-invulnerable. No one would mistake their handling for a Leica---they're tractors to Leica's BMW, but in a rough environment, you might prefer to drive a tractor rather than a BMW! There are complaints about them, but image quality isn't usually one (unless you get a lens that's just plain misaligned). They might be worth looking at in your position. Their 35mm lens (which I believe is the length your Olympus has) is a real winner, a clone of the prewar Zeiss Biogon, but not collapsible---are there any collapsible 35mm lenses? All the ones I know of are 50s, but I'm not an expert.