This is not normal - something is not right.

Mine, with a standard baseboard mount, is quite solid. It shouldn't be necessary to wall mount it just to achieve basic stability.