I know what you mean. Most of my prints are 20x24. There is something definitely different about a print that is 20x24 and one that is 8x10. The are not the same print. So I think that women are right, size does matter.
It may have to do with how a picture is viewed. An 8x10 is usually viewed at about 4-5 feet away and a 20x24 at about 6-10 feet away. Perhaps there is a stronger impression because of the size and the viewing distance.
I'm also cognisant of the line that "if you can't make it good, make it big" but a bad picture in 8x10 rarely gets much better in 20x24.