Ilford SFX has big grain, close to Delta 3200. The only time that you don't "see" grain is when the film is 4x5, and that's just a factor of enlargement. You could try a couple of different developers, and see if something works better for you. However, if you like the IR effect, then 10 rolls is the minimum learning curve. When I started out with Kodak HIE, I roasted at least five rolls before I knew that my changing bag wasn't IR tight. (I didn't have any dark space, and I loaded my film onto reels inside a changing tent.)
I mostly use Xtol, and some D-76 or Ilfosol. I haven't tried pyro yet with that film.