My vote is that if you can't stand the covering crumbling off, then simply take the door off and clean it. This would also be an excellent time to replace seals if needed too. I have one N90 that I have left the door as is with it's covering sticky and barely coming off and another N90 that I have removed the covering from. These cameras are not my go to cameras (my F5, FA and F4S are in that order now), so outside appearance isn't that big of a player for me for these cameras.
My experience is similar to yours performance wise for the N90......perfect, infocus results. I love using them which reminds me I should probably run a roll of film through them....it's been awhile.