Lets look at it this way.....
Bleach III for C41 is used as is. No dilution. That is extreme but is usable for an illustration.
Now, Imagine a fix that is the same, requiring no dilution for use.
If you mix them together, you are diluting them 1:1 with each other and you slow things down.
This is an extreme example of the case. What really emphasizes this is when you surround the silver metal with dye clouds and when the silver has all kinds of "grunge" on the surface that must be scrubbed off before it can be bleached.
Analysis shows that out of the 300 mg or so per square foot, less than 10 mg per square foot should remain in order to get good grain and color. The leftover silver metal adds to grain and also darkens color. The OP has pointed that out to us. Well, in E6 you can also see a slight distortion in whites due to silver and silver sulfide.
Mike is correct, in that ferricyanide does a good job removing silver metal. However, IDK if current dyes are tested for image stability with ferricyanide. It is a strong oxidant. Oxidants are bad for some dyes.
I am, however, speaking from the POV of one who ran image stability and bleaching/blixing tests for over 5 years, and who took the ICIS short course in image stability.
In the long run, if you are happy, why worry?