I never got too far into the Japanese fixed-lens rangefinder world, but it seemed like if you were willing to forgo the really fast lenses they got a lot cheaper. The Canonet 28 seemed like a good camera that was basically being given away, even when prices of the f/1.7 models were going kind of nuts a few years ago, and the other makers had similar sub-top-of-the-line models. Those might make a decent very cheap intro to the rangefinder gestalt.
Some of them have no manual controls, though, and they were usually designed to use mercury batteries. My Minolta worked fine with a modern battery of slightly different voltage, so I guess it had some voltage compensation built in, but not all models did. The difference might be livable with print film.