The time got longer because you made the image larger - effectively you had more bellows-factor, except in this case the "bellows" is the gap between enlarger lens and paper. Or stated alternately, you had to get the same amount of light from a much smaller part of the negative, so it took longer.