The problem is likely that you're not toning. Albumen prints must be toned. If gold toning is too costly, you can get not unreasonable results from selenium. I find the tone a bit more neutral with selenium than with gold, and it seems to serve a protective function. I have selenium toned albumen prints that have been stable for several years.

When you start gold toning, you need to plan your printing sessions around the toner, because it only lasts for one session, at least with the formulation I've used.