Sounds to me like Daniel is probably right in that it is a diopter problem and what you have doesn't work with either focal in your bifocals.
I need regular reading glasses. If I put a +1 diopter in my pop up magnifier I can focus just fine with out my glasses. However if I want to focus with my glasses on
I have to have a regular not +1 magnifier.... then I can focus through the magnifier even without putting it right up to my eye.
You should be able with the correct focus for your eyes to easily focus your Rollei.
You can get a Hasselblad chimney with adjustable diopter for a lot less money than the Rollei Chimney however with an E you can't put it on your camera. So I think whatever diopter you have in place in your pop up magnifier, you need something else.
Dennis