More weeks passed...
Alcohol is definitely the bad guy here. In an experiment I added some pure alcohol to a well mixed 10% solution, and dichromate crystals started dropping to the bottom of the container after few seconds.

After switching to acetone, my solutions are still bright orange after a couple of months.