I have found it necessary to put a level directly on the ground glass, or on the ground-glass frame nearby, to ensure that the film plane is vertical.
This is what governs the convergence you see.
However, I have sometimes pulled my head out from the dark cloth and found that the building was tilting in reality.