If the camera is in the pack or on me depends on my camera. When i was hiking with a slr I had it packed into one of those smallish camera bags and that was in front of me. My 4x5 (which i do backpack with) stays in the pack unless I am using it.
I say just bring the 50mm and your cannon ae-1 w/o any accessories (definitely no tripod). Weight wise its the right idea. I tell my self that if I can't make an image with a 50 n' body then its not worth taking. I'm here for the hiking, friendship, and views. I know if I want a stunning image I need to work it, experiment, and possibly revisit it multiple times. You just don't have time for that when your hiking. Save your self the frustration and oz's, and take the bare minimum.