Actually, in that case it sounds as though there is solid mechanical retention, so perhaps three or four tiny drops of white glue carefully applied with the end of a toothpick and evenly spaced around the circumference might be enough to keep the glass from rattling around. Using it thicker would reduce the chance of it flowing somewhere not desired. But like AgX, I'm not sure being absolutely tight is that critical, assuming there is no way for it to fall out.
I'll give the white glue a shot. The looseness isn't a problem until I try to clean the thing...and then it becomes infuriating. I'll post some results when I get them.