I've read that minimal sulfite is better for the stain, but a certain amount is required if the fixer is to be reused. Otherwise it will deteriorate very quickly. For one-shot use, you could leave it out.
For paper, some sulfite is recommended to preserve the thiosulate. It's harder to detect insufficient fixing with paper.