You're probably seeing a combination of carryover and reduction of stain.

Disparagements regarding the current faddishness of alkaline fixers notwithstanding, I would just note that if you switch to an alkaline fixer, you won't need to worry about overfixing or reduction of stain in the fixer.