When projection enlarging, you have light dispersal and reflection being caused by the grains of the film and the lens elements that causes photons to intrude into image forming areas on the paper where they normally would not go. You're allowing light to be bent away from a straight line, over a relatively great distance. Yes, the same grain of silver in the piece of film is going to reflect random photons at the same angle when projection enlarging as when contact printing, but the photon is going to go anywhere from millimeters to inches off target when projecting, but only fractions of a fraction of a millimeter off-target when contact printing, because the emulsion, containing the image forming silver, is in direct contact with the image-receiving silver (or other material) on the paper. This is the same reason why you need to boost contrast filtration when printing larger - an 8x10 print of a given negative may print perfectly at grade 2, but at a 16x20 it may require grade 3 to look the same.