Within a family of films, lower speed films will have finer grain and tend to have higher resolving power. This means detail and image structure will tend to hold up better as the enlargement size increases (and with lower graininess). Increased graininess will also tend to be most visible in images that have expanses of featureless tonality (skies for example) rather than in highly detailed, "busy" images. Contrast also plays a key role.
Those are the scientific facts. But then there are the subjective aspects - ie there are no rules governing the maximum size of the enlargement you choose to make from any film of any speed. It all depends on the subject matter, your own preferences, planned viewing distances etc.