It is possible that the focal length of the magnifying glass is too small for the speed graphic. Probably around 50mm.

If you find yourself in a shop with a variety of magnifying glasses available, try them out by focusing something bright like the light fittings or the shop window onto the palm of your hand to judge the necessary distance it needs to be away from the film.