As others have said, the board movement is only an issue if it moves during the exposure, or it moves so much that there is a light leak. If the board is a Calumet board meant for the camera the light trap built into the board or mounting should take care of any light leak problem. If it really bothers you use some tape or shim it some at one edge. Don't shim behind the board though, or you could introduce a light leak. In practice, slight movement is generally no consequence.
If the holder isn't seated in the ridges and groves you will get light leaks, but if it is inserted correctly there is no problem, that's why the ridges are there. Beyond that, the spring tension holds the holder tightly in place. Just make sure you insert the holder fully. When it is, you will see that it's pressed tightly to the camera top and bottom.

For positioning, the critical dimension is the distance from the top surface of the holder to the film plane. Everything is designed so that the ground class sits at that same distance. Some very old cameras, required holders that were specific to the camera, but in the 4x5 realm, by the time your Calumet was made all that got sorted out and standardized, and had been for many years.
If you encounter wood holders, just make sure they are not warped. Generally the plastic holders are pretty robust.

The interesting thing about view cameras is that since everything is made to move in just about every plane anyway, tight precision though nice, is not a requirement. As long as everything can be locked so that it doesn't move at the wrong time, it doesn't matter how much movement there is otherwise.