I have used a largish lumbar pack in conjunction with a fly-fishing vest for a couple of decades now. The lumbar pack has a shoulder strap as well, so never has to touch the ground. I simply unbuckle the hip belt and use it slung to one side, strap cross-body, and work directly from it.

I find this set-up allows me more freedom to scramble, climb and hike and is more balanced than a backpack.