Unless there is some severe mis-alignment of the lensboard, or you are not mounting the lens properly (unlikely), I would suspect a problem with the focus screen. You need to check the focus at the film gate and the screen with both lenses using a piece of ground glass or some taut transluscent sticky tape.
What type of focus aid do you have? If it is a microprism or split image type, what is the number on the screen? You will have to demount the screen to see the label. There are some screens that are designed to work well with the longer lenses.
I find it interesting that the 180 is giving better results than the 80mm. I would have expected problem to show up the other way.
The parallax setting on the side of the camera has no effect on the focus, by the way.