Depends on the angle of view you want (or can support if you're modifying a regular camera). There's a calculator on the Mr. Pinhole site that might help (not to mention other info.)

I used about 0.36 mm with about 90 mm film to pinhole for my SQ-Hole 6x6.

Pinhole Designer is a Windoze program that can be used to calculate a bunch of stuff too.