I do not believe in the formulae for the holes there are more than 50 formulas for that hole. All these formulas give very differente result with a sometime a factor two betweens the results. I wrote an article in my blog with formula from various great specialists of optic, physics etc which give totally deferent results. The article is in franch, but you can have a look at the fomulas and the curves)
But saying that, that does not means the size of the hole is not important. If you want a sharper results, 0.4 mm for a focal length of 51 mm seems to me a bit too big. A hole with a diameter of 0.2 or 0.3 mm should improve clearly the results in term of sharpness.
Second thing, you said that you used a part of a Chinese watch. How thin is the material. I doute it is a thin as for example a piece of beverage can. The thickness is clearly also important (an taken in account in the formulas).