Erik, when I started my 1:1 stuff, locking the bellows on an LF camera to ensure true 1:1, I was immediately ocnfonted with the fact that only the objects in the plane of focus will be 1:1. Hence we perceive, for good reason, that objects outside the plane of focus (= out of focus elements) are necessarily rendered at some other scale. As for how this works with the eyeball, it is something trickly about the brain that we perceive infinite DOF when we look at things, but... of course, the eye is ultimately just a kind of lens which does have finite DOF.

There are many interesting perceptual oddities, not just in sight but analogously in sound as well.