TF-5 is nearly neutral. In fact, it is the most nearly neutral of all B&W fixers.
Going directly into the fix though carries developer into the fix which is not good, throw away or not! You have to remove as much of this junk as possible before you go into the fix so that the fixing action takes place in the absence of HQ, Phenidone, Metol and whatever else could be in there.
The changes in pH have been there for ages. Early processes used a stop and had no real problems. Going from developer to acid stop to alkaline fixer certainly changes swell, but I have tested it and it works even with TF-4. And, no matter what you do, you will have changes in swell due to the ionic strength of the solution (or just water) and pH changes. After all, a ph 10 developer followed by a pH 6 city water rinse followed by a pH 8 TF-4 or a pH 4.5 Acid Fix certainly does not hurt anything.