What you gain in cost of the lens you will loose in cost of another body (and more lenses for it).
-The Nikkor AIS 28mm f2.8 has a bit of a cult status because of its CRC and excellent reviews. But even then, I have seen it for relatively decent prices if you look hard enough.
As an alternative, the Nikkor 24mm f2.8, AI 28mm f2.8, or 28mm f3.5 go for much less, and are excellent lenses just without the CRC.
The Olympus OM lenses you reference do not have CRC either, so why not just stick to your one system?
And lastly, other highly recommended lenses are the Kiron (aka Vivitar serial numbers 22*) 28mm and Sigma 28mm lenses. The sigma is a real bargain.