Pick one to standardize on in your darkroom. It doesn't have to be perfect. Just reasonably close. Checking against a fever thermometer is a really good starting point, especially for color temps (because it will be most accurate at or near its target of 98.6F). Once you have established your standard, then calibrate (measure the difference) for all your others. Note the differences and keep a small chart on the wall for reference.
In my darkroom my standard is a Hass Intellifaucet, since this supplies all of my mixing, tempering, and washing water.* I run it continuously to fill a large plastic container, then all my other thermometers go into that container (partially or fully submerged, per their design) and I check the offsets. Then I keep a reference list that says,
Kodak Darkroom: 68F = 69.6F
Kodak Tray: 68F = 69.0F
and so on. If I needed temperature set points other than 68F I would repeat the procedure for those specific points, since the thermometers are likely non-linear in their response.
[Edit: If you really have the need or desire to calibrate all of your darkroom thermometers to a more accurate source than an ice bath, fever thermometer, or Intellifaucet, then check out these NIST traceable fractional degree calibrated mercury and calibrated non-mercury instruments offered by ICL Calibration Laboratories, Inc. Definitely not for the faint of wallet. Note that the uncalibrated versions are, however, quite reasonable.]
* The product description for the Intellifaucet states:
"Each Intellifaucet is tested and calibrated to +/- 0.1 F at two widely separated temperatures. Reference thermometers are traceable to the National Institute of Standards and Technology."
and that's good enough for the non-analytical work we all do in a darkroom. Consistency is far more important than absolute accuracy.