Your pictures are very soft. Pinhole pictures are always a bit soft - and maybe this is what you want?
But if you want sharper - there is a mathematical relationship between the hole to film distance and the diameter.
The idea is that the circle of confusion gets smaller as the diameter decreases, but as the diameter decreases diffraction gets worse, so there will be some ideal compromise diameter for best sharpness.
If you use the formula d = SQR( F/k)
where F = film to hole distance, and k is a constant, about 1300.
(SQR means square root)
Different people use different constants and formulas, but it gets you in the right area.
For a film to hole distance of 50mm - you want about a hole diameter of about 0.2 mm, I reckon.
Or... to work that backwards from a hole size of 0.4 mm, F = (0.4*0.4) * 1300 = 208.
So, I reckon that pinhole is ideal for being about 200 mm from the film. (If I've got my maths right?!!!)
What size film are you using? That would be a 'Normal lens' on 5 X 4"