Quinten - -
As the degree of enlargement increases, both the grain in the film and any abberations or deficiencies in the optical system become more noticeable. A 4x5 print from a 35mm negative made with a 50mm lens can look very sharp, but if you make an 11x14 enlargement from the same negative, and using the same enlarger and lens, you will see more grain, and any softness in the resolution of the lens will become more obvious. Likewise, any problems with alignment will be more visible.
The other issue here is viewing distance. You will always see more grain and optical defects if you are viewing a print from a close distance than you will if the print is further away.
I've made cropped 11x14 prints from 35mm negatvies using a 50mm el Nikor lens where the full frame enlargement was more like 16x20 inches. They weren't the same quality that I get from a 4x5 negative, but they weren't all that bad either.
Kodak used to brag about making an enlargement for the 18 x 60 foot Colorama display in Grand Central Station in New York from a 35mm Kodachrome slide original. Now that's serious enlargement.
There is no absolute rule on how much enlargement you can squeeze out of a negative - as the artist, you have to decide when the optical defects take away from the content of the image.