Regarding converting a meter reading to one's pinhole camera's f-stop:

(Fc/Fm)^2 * Tm = Tc

Fc = f-stop of camera
Fm = f-stop used on meter
Tm = exposure time recomended by meter
Tc = converted exposure time

In your example, Grainy, I didn't see where you gave us the f-number of the pinhole camera, but let's guess at a number like f/200. So here's the hypothetical calculation for the values you provided:
(200/22)^2 * 4 = 330 seconds, which is 5-1/2 minutes.

Provided that you keep using the F/22 reading on your light meter, and provided that the f-stop of your camera doesn't change, you can perform part of this calculation ahead of time, thus:
(200/22)^2 = 82.6

So, you would write down this multiplier of 82.6, take it with your camera, and multiply it by the meter reading (in seconds) to arrive at your corrected exposure time. Keep in mind that this formula does not account for reciprocity failure. In my experience, paper has very little issues, whereas film does.

NOTE 1: In the above example I assumed your camera's f-number was 200. You need to measure the diameter of your camera's pinhole, in millimeters, and divide that number into the camera's focal length, also in millimeters, to arrive at an accurate figure for your camera's actual f-number. Then, plug that number into the formula to get your actual working correction factor.

NOTE 2: I'm assuming that you're shooting paper negatives in daylight illuminated scenes. Remember that paper is, for the most part, only sensitive to UV and blue, and a very slight amount of green. Thus, if you meter an indoor scene that's illuminated by artificial lighting, the meter reading will not be accurate (because the meter's spectral response is much wider than the paper's), and you'll have to manually figure out how much extra exposure to use. For this reason, I recommend using panchromatic film (rather than paper) for artificially lit scenes.

NOTE 3: I've found that for using paper negatives you should try to be more exact with developer temperature and dilution (and freshness) than you might when using paper for prints. For instance, I use Freestyle's Arista brand grade 2 RC paper for negatives. I use Ilford's Universal paper developer, freshly mixed at a 1+15 dilution, and develop the negatives with the chemistry at 68f. That being said, once you start using the formula for converting your meter reading, and once your development process is consistent, then you should run a series of calibration tests to find out your paper's actual working exposure index; the value of "6" you provided may or may not be accurate.

To do these tests, mix up your chemistry at the proper dilution and temperature, in the daytime, and then shoot a series of exposures of a day lit outdoor scene that has a mix of shadows and highlights, using different exposure indexes for each shot. Start at about EI=2, working up to EI=12, making the calculation from the above formula for each shot. Write the metered exposure index on the back of each paper negative with a Sharpie marker. When you're done processing the negatives, you'll be able to review them and easily figure out the best exposed negative, and its corresponding exposure index.

Good luck, keep us posted.