If it's important to you, you'll find the time. It may help if you break down your photo endeavors into blocks of time. While I stay out of the darkroom unless I have a 3-4 hour window, I've found many things I can accomplish with smaller blocks of time:
If I have 10 minutes, I'll do some print spotting.
If I have 20 minutes, I can shoot some still life stuff. (when time is rare, I leave the camera setup, and ready to go, in a corner.)
If I have 30 minutes, I'll do some hand-coloring, or matting/mounting.
If I have an hour, I can shoot a roll at a favorite spot of mine, which is a 5 minute drive from my house.
It's all about balance, and utilizing the limited time you have. Sometimes I have less than 10 minutes, and I'll find myself looking into APUG's Gallery, finding inspiration and motivation.