Far from being an expert myself, I decided to go with bulk-loaded rolls of Ilford Delta-400. Here in Quebec City, a roll of 24 exp was $6.50. I went onto eBay, found a bulk-loader and 10 (new) Kodak cassettes for $25. 30m of Delta 400 goes for $55.
I went with Delta-400 because it is my favourite film in terms of flexibility. Should I need a roll of something else, I'll go and pick one up.
Long story short, I did the math and bulk-loading works for me!
As for paper and chemicals, I have a couple of choices in town, and all of it is fairly reasonable price-wise. Again, I am no expert, but I would imagine that apart from bulk-loading, there are probably no really significant ways of saving money.