How about a shim with a small pieces of aluminum foil or plastic wedged into place with a toothpick. No chance of messy glue and reversible if not satisfactory. Or as Steve says. I've had a filter or two that rattled and never affected the photographs.

http://www.jeffreyglasser.com/