With the XA series (excepting the XA1, of course) it is imperative to use only high-quality Silver Oxide batteries. Any lesser batteries will result in under-exposed shots. This is why some feel their photos are dull and lifeless. Because they are underexposed.
The XA shutter works with electromagnets that hold the shutter open. The power requirements are high enough that alkaline batteries (especially those 10 for a dollar that are so appealing to buy) can't get the job done. The Silver Oxide batteries are the only ones with enough power to correctly work the shutter.
Get yourself some proper batteries and see if you aren't much happier with your XA photos.