Check the accuracy specs on that thermometer. I'll bet it is supposed to be accurate to ±1º C. with a resolution of 0.1º C.
That means the thermometer can be off by one degree Celsius in either direction. If you are trying to read 32º F. the thermometer can display a temperature between 30.2º F. and 33.8 F. and still be within spec. (Remember, virtually all electronic, digital thermometers actually read the temperature in Celsius then convert to Fahrenheit.)
With that in mind, your thermometer seems to be in spec. The temperature readings you are getting are within its range of accuracy.
This does not mean that your thermometer can't be dialed in better. It just means that you might need to adjust your expectations a little.
A good analogy would be like if I gave you a rifle and told you that it is capable of shooting ten bullets into a one inch circle at a distance of 100 yards. Then you shoot the gun and report back to me that the gun only hit a half-inch bullseye five times. Yes, if you aim very carefully, you are likely to hit a bullseye smaller than an inch but the gun was only designed to shoot into a one inch circle.
If you want really good accuracy you should probably invest in a laboratory grade thermometer but, if you are the tinkering kind there is a series of articles from the magazine, Scientific American, that give some good information on how to build and use a triple point cell to calibrate a thermometer. Do check them out!