You're going to hate me for this answer however, in a previous life I had the experience of photographing the occasional world cup downhill ski races. Along with the video crews it was not uncommon to ski/side-slip the course handholding the 400/F2.8 or 500/F4 with a backpack of film, spare cameras, crampons etc. That certainly focused the mind on the ski-ing!
Being slightly more pertinent; having seen those types of lenses take a tuumble and having a tumble with a back pack of gear - providing the accident speed was not too high there was more danger trauma from getting snow wedged on a lens than the tumble. High speed crash, different story.
Key point being, anything that detracts from your ski ability is a danger. A novice could break bones with a point and shoot, a skilled skier could take a 5x4.
Try a bum bag (fanny pack in usa?) instead of a rucsack.

Sim2.