Hi Casey, and welcome to APUG. I'm also from the east side of Melbourne, but not quite as far out as you. Did you know that there is a Melbourne APUG get-together on May 27? You'll find the details under the thread "Melbourne APUG get together - MAY 2012".
I also have an FE2, along with an FM2N, an FE, a couple of FMs, and several other Nikon bodies. The FE2 is one of my favourites. With regard to batteries for the FE2, you might prefer the silver SR44 which has a longer life than the alkaline LR44. Both of these are also known as S76 and L76 respectively. However, instead of two of those I prefer to use a single lithium 2L76, also known as CR1/3N which has the longest life of all. They are all pretty easily obtainable. The one drawback of the FE2, and the FE, is that it is battery-dependent for its shutter operation and you could get caught short in the field if the battery goes flat. Simple solution: carry a spare battery.