Correct rule of thumb, incorrect reason. The right reason is that lenses designed for general use are optimized for a large subject in front of the lens and a small image behind it. These lenses should be reversed when used at magnifications greater than 1:1 (small subject in front, large image behind) to preserve their optimizations.
And my understanding is with asymmetrical lenses those optimisations include field flatness/spherical aberration. Is this incorrect?