Not as small as an XA or a Rollei 35, still very small and most of them with full manual control and decent range of shutter speeds and aperture:

http://www.cameraquest.com/com35s.htm

As an expensive and high-quality alternative, a Contax G2 with the 45/2 pancake lens.

Or, as has been said, a Leica III with a collapsible lens.