System cameras and young children don't go together. They lack the motor skills and unless they're of an unusually boffinish persuasion, the kick is getting the pictures, not twiddling with knobs and dials. They also get fingers on the lens all the time, so good glass isn't a great idea. I'd go for a 1990s P&S, there are lots of them and they cost buttons, and have fun developing the film afterwards. Buy an old 35mm enlarger and take over the bathroom one night a week and show them the magic of the development trays.

Get them an SLR when they're older, something like a Canon AT-1, light to carry, fully manual and with batteries you can still buy and lenses that won't break the bank.