Even though the 28mm seems to focus accurately, I'd still consider a camera body problem (which BTW?): misaligned mirror or badly seated focusing screen.
Try to beg, borrow or steal a longer lens, which will make testing easier.

The AI conversion shouldn't make the lens "shift", whatever you mean exactly by that. If there is any play in the lens mount (loose mount screws or worn flanges), that could also cause the problem.