The commercial mixes all contain a sequestering agent (Calgon, EDTA or similar) for precisely this reason.

I wouldn't expect a hint of gypsum to have any serious effects on the longevity, no. After all there's already a layer of barium sulfate in there (baryta)!