There are three factors, all of which contribute to losing light at the corners.
The first is the difference in distance between the center and edge or corners. This has been pretty well covered here so far, and I think is fairly easily understood if one has a practical understanding of the inverse square. Practically, you can visualize the appearance of the hole at different distances. The hole will appear to be the largest when it's near (along the axis). As the hole is moved away, it will appear smaller. At 2x the "f" distance, the hole will appear to be half the diameter it seemed to be at "f". Because the quantity of light is dependent upon the AREA of the hole, this is 1/4 the light, as Vaughn said.
The second is change in the aspect of the hole as seen from points which are away from the axis. The hole becomes apparently elliptical, rather than circular. The effect can be easily demonstrated by punching a circular hole in a note card and looking through it both on the central axis and tilted to approximate the view of the hole at the corner of the camera you are modeling. The difference in AREA between that ellipse and and a circle of the same size would account for the loss of light. This effect is very substantial, so the amount of light available at the 2*f is actually a great deal less than the 1/4 the simple inverse square would suggest.
Third, pinholes such as the common laser drilled examples available on commercially made pinhole cameras are actually TUBES, and this is where the question regarding the thickness of the material mentioned above by DLM comes in. In lens terminology, this is "cutoff". This can be visualized by holding a tube, such as the ones on which toilet paper is served. Look directly down the center. The opening is round. Tilt it a bit, and you will see how the front end of the tube on one side, and the back end of the tube on the other, begin to encroach upon the circle. Again, the difference in AREA of the opening between the obstructed hole and the unobstructed hole gives an idea of the amount of loss, which also can be substantial. Of course, the toilet paper tube is greatly exaggerated, but I hope you get the idea. ANY drilled hole, regardless of the fantastic technology employed to drill it, will have this tubular character. For this reason, it is to our advantage to use the thinnest possible material and make it even thinner if we can. The best pinhole would be one made of material with no thickness at all, which isn't possible. The next best would be a hole made with a sharpened edge all around, like a knife. This probably isn't possible either.
So, the OP's question is not one that can satisfied with a simple answer.