If you are looking for a fixed lens, rigid, relatively simple camera, with a good lens, there isn't a lot between basic box cameras with no adjustments and coupled-rangefinder or reflex cameras that have lots of sophistication.

You may want to re-consider a folder, for example, the little Welta Weltax I had up for sale (and sold) recently was a very pleasant little camera to use and has a pretty decent lens.

Another option might be something like a Yashicamat TLR, and just use the sports finder most of the time. They are light, reasonably compact, good optics, and don't cost a fortune.