I use a six inch set of Ilford filters under the lens in a 12x12 cold light Durst 138S conversion for 8x10. 4x5 negatives and smaller go through a dial in variable contrast Saunders 4550XLG/VCCE Enlarger.
I use the big Ilford filters above the condensor in my D2V. Since the filters are just below the light and far above the negative, they'd have to be downright filthy before the dirt would cause a problem. The Ilford filters are one of the very few things I bought new in my darkroom, and I've never regretted the investment.
I'm sure that under-the-lens filters could be fairly dusty before they would cause a problem, but I know I'd obsess about keeping them clean.
I use both a color head and Ilford filters on my 2 1/4 enlarger. Recently found a brand new set of 150mm Kodak filters (PolyMax II) at a local store for $30 for my D3 4 X 5 enlarger (condenser head with a filter "drawer"). Rebuilding a color head for this unit and will then see which way I go.
Like the filters more than the color head but the later is easier to use.
I use the Ilford filters above the condenser. In high school, we used the ones below the lens and the smallest scratches showed up very easily. I could tell which filter I used on which print just by knowing the scratches.
I have a set of Ilford filters that fit the filter drawer of the enlarger but I've just about quit using them. For the most part, my negatives print well without filtration. I've never used those that fit below the enlarging lens. My previous enlarger had no filter drawer but it was easy to pop off the head and place them on the condenser.