The simplest answer is that larger grains are faster and in film there is a range of grain sizes. So the shadows will have the greatest effect on only those faster grains, hence more visible grain.

This is also why reversal processed films have much finer grain. The larger grains are used in the initial exposure and unfortunately we usually fix away the finest; but in reversal processing the final image is composed of only the crème de la crème .