Aggie, like Frank I have heard of a similar story. A (female) photographer who had to stop all activity in the darkroom because of bad hand sores due to the chemicals. She never had any trouble before, the sores suddenly developed. She was told that such an allergy can "build up" in the body, and suddenly become a nuisance. So the fact that you don't have any skin problems at the moment, may not mean that you are safe for the future.

I only use gloves with aggressive chemicals like selenium toner, and when I am warned that the chemicals will stain my fingers. Mostly I just work bare handed and use tongs on RC paper, bare hands on fiber. I _do_ very frequently rinse my hands, even in between touching the developer, stopbath and fixer, and like you I wash them thouroughly after a darkroom session, and use a _good_ hand cream.
The worst I ever suffered from are chapped hands.