In response to your question about jazz, in PLAYING jazz (and I suppose this is different than listening to it) a lot of the challenge/pleasure/excitement comes from the interaction of the musicians. Thus, playing a standard gives a structure (key, changes, basic melody) to the ensemble for the soloist to improvise over. An entirely new song could do the same thing, but all of the musicians would have to learn it before they can play it. The ensemble improvisition is the act of creation, so yes, improvisation is that important.
It all comes down to creating art within a structure, and photography is the same way. (I don't believe one can create art without first devising some kind of structure to work within.) I think Adams was trying to create, or maybe visualize, such a structure when he devised the zone system. It's a systematic way of thinking about the structure of photography - film speed, light, development - in his words, writing the score.
I see a lot of similarity between jazz musicians interacting with one another and a photographer interacting with light.