Unfortunately what you lose is that look that fiber paper has. My B&W prints are on glossy fiber because of the look of the surface. I find that glossy RC paper is too glossy and smooth it has that plastic look - it's basically ugly.
I see your point about the commercial viability of color fiber paper - I was thinking about it for people who make their own color prints. However, since there are far fewer color printers than B&W printers, I can see why there would be no money in it.