I am fond of styrofoam coolers and aquarium heaters to keep bottles or jugs of chemicals warm.

I have also used plastic sandwich bags filled either with ice cubes (to cool) or hot water (to warm) beakers of chemicals by immersing the plastic bags in the chemicals. Just need to make sure that you have separate marked bags for the various chemicals.