The ME shutter will work without the battery, but only on 1/100 or B settings. Quite a neat design really, 1/100 is the x-sync speed so you can either use it indoors with flash or Sunny 16 outside if your batteries expire.

I'd check the battery cover plate, chances are it's got some gunk from the old battery on it. Just clean the part where the battery touches it with very fine wire wool (make sure no shreds of that get into the camera) and reassemble. I resurrected a "spares or repairs" ME this way last week.