I think most toy cameras are made from plastic, and have a plastic lens (although I'm not sure if the Holga with the glass lens would still be considered a toy camera). There may be exceptions to this rule. In some photography competitions, pinhole cameras and lensbaby lenses are also acceptable, but disposable cameras and old (vintage) cameras like Brownies are not.
My favorite thing is to go where I've never been. D. Arbus
North America just north of that sharp right turn North America makes on the Atlantic coast.
A toy camera is any camera you would be willing to hand over to an unsupervised child and not worry about the child or the camera being harmed. I would not put the Holga into that category, I don't believe that a Holga could take the punishment a child can dish out.
Fisher Price made a lot of really great cameras for kids over the years.
You would be surprised what passes for "toy" at my house, my kid has a Kodak cameo Motor EX (possibly the last folding camera Kodak ever made, it has a plastic folding bellows), a Canon WP-1, and a National Geographic half frame binocular camera that I let her run free with. She also has a Nikon N65, that she only uses under parental supervision. As for my equipment, she is only allowed to use the cameras that she can hold by herself and look through the viewfinder, the only cameras she can't use now are the F4 and the Pentax 6X7, and always supervised.
"Would you like it if someone that painted in oils told you that you were not making portraits because you were using a camera?"
"Shouldn't it be more about the joy of producing and viewing the photo than what you paid for the camera?"
They don't necessarily have to be toys, and there's a question of whether Dianas and Holgas were literally intended originally to be toys or just as inexpensive cameras manufactured from cheap materials, but I think there's an aesthetic there. The idea is to make an interesting photograph with a camera that doesn't necessarily give the user much technical control and may even introduce artifacts that are beyond the user's control. It's about giving chance a greater role in the production of art.
All my cameras are my toys and may be that is why I never have backup camera. I have many cameras but never a backup. If one of them quit working I would stop shooting and cry. I can't see myself when one of mine quit working simply pull out the spare and go on. That would be too professional like when I lose or break my tools at work. I don't care about the shoot I care more for my toys.