Is it possible that your timer varies a little bit? It depends on how many cycles of a clock count it uses to count seconds and would have more effect on short times than long.

I made a timer a few years ago and made the mistake of using a free running one second oscillator to advance the counter. At the time of pressing the start button, the next pulse could be anything from immediate to one second away so it had an accuracy of +/- one second. Linking the start switch to a clock reset would have cured this but I didn't get round to doing that!

If it is the mains voltage, would it be possible to replace the bulb with a low voltage type and use a regulated dc power supply?