I have an 0.6 ND filter (2 stops) that I bought for taking pictures of waterfalls, but lately I have been using it in the darkroom a lot. It really isn't unusual to have short times at f/22 when making 8x10's.
You say you have four lenses, what other lenses do you have? I have an idea that might work - and it will cost you nothing!
If you have a longer lens, like a 135mm then you could raise the head of the enlarger. This would give you less light, and more reasonable exposure times.
If this works, you can save the 80mm for when you are doing 11x14 or 16x20...