When I shoot with the Rollei's, I keep my glasses on and use the flip magnifier. I have progressive lenses, and have learned to find the sweet spot in the glasses (but the magnifier has to be all the way flipped up).
Same with 35mm, find the sweet spot.
4x5, is different - I compose with glasses off (I'm somewhat nearsighted, so not too bad, plus I'm composing general forms and light, not texture, etc so I kind of like it). I focus with a Scheider 4x on the GG.