Here's the simplified answer, I own both systems, and practically have the same setups you are looking to get. I like my Olympus's, and frequently grab them over the nikons. Both have very good image quality, but I would have to give it to the zuikos. The size and weight make quite a difference too, the more compact bodies and lenses make them a pleasure to carry and use.
It is hard to recommend one over the other as many people have their own personal preferences, and I am sure many will chime in on both sides for either one. Best bet is to try them both, pick a body, and get a 50mm for it, shoot them both at the same time, decided and sell the other. Dont be temped and fall into the gas trap and build up both systems, it gets expensive very quickly.