If you want fine grain you really should try Ilford Perceptol. My favorite developer. Dilute it 1+2 or 1+3 to get more acutance, otherwise you may not achieve the sharpness that you desired. I use it with FP4+ and HP5+, but it will work very well with PanF+ too from what I´ve read. Delta 100 and Perceptol should also be worth a try. The difference in grain may however be very small between diluted Perceptol and ID11 or Xtol, but it should be recognizeable in the large prints you want to make. Perceptol cuts the film speed in half and comes as powder, so it is not as convenient as other developers, but no black magic either.