Like a magnetic tape recording that fades over time, they lose their latent image over the years. It ends up like they were underexposed. That plus base fog has also built up so you need to develop your image above the base fog.
But part of what you're saying is true.
I once found a half-used 12 year old film in a camera and shot the rest of the roll before sending it out. The original shots were diminished from new but still pretty good. The new shots were all messed up.