This is an excellent idea. I have considered doing something similar, and I may actually begin to move in this direction, perhaps getting down to two bodies (one color, one b&w). With my five bodies, things can get complicated and confusing.
I would agree that 800-speed film is not necessary. The highest I go is 400, and, with very rare exception, it has always served me well. I like Kodak Portra 400NC and Ilford Delta 400.