Yes, the prism needs to be coupled to the lens aperture lever or the mechanism will not engage and thus potentially snap the coupling chain — an involved and extremely fiddly repair job that is a bit like repairing a watch.
Process is: prism off, then lens. Replace the prism, then re-mount the lens. Ensure the prism coupling pin engages positively with the shutter speed dial. If the prism has a switch, turn it on to verify the meter is reading at the selected aperture/shutter speed value (if in low light, switch to bulb and maximum aperture of lens: needle should shoot up to the top). If the prism has been correctly fitted/coupled, it is safe to change just lenses any time you wish.
The prism is generally only removed to facilitate cleaning of the focusing screen.