While the distance to the light source can affect sharpness it really can't affect it to the degree where you could easily see it. The most OOF you could get with a diffused light source is about the thickness of the negative, and also, the effect wouldn't be limited to certain areas. In practical terms the distance and size of the light source are of little consequence, as long as it is far enough away to be even. Your issue is most likely a contact issue, or possibly an issue with the sharpness of the negative image.
The contact being made in the frame is the most likely culprit. Check the glass and the back for flatness. No amount of pressure can make up for a defect in those areas.
EDIT: I see in re-reading your post that you aren't using a frame. What are you using under the glass neg paper sandwich? That is likely the problem. You should consider a good quality frame, if possible.