The orange indicator with the circle should move when the aperture or speed dial is turned. The white needle will react to varying light (assuming your finger is not over the meter sensor - a common issue).
The battery required is a 625 1.35volt mercury cell - no longer available as you may already know. I use a 1.5volt 357 silver oxide cell in a converter made from a diode and 625 case.
There should be a silver insert (cover) that screws in over the battery and supports the end of the film cartridge. Other than a parts camera, I don't know where you'd find a replacement.