I am going to echo the suggestion of the Fuji GA series. I have one and it really is pretty much a point and shoot camera. In size it feels like a mid-range SLR with a pancake lens attached. In operation I can work with it about as fast as I can work with a 35mm compact, though with the benefit of a high quality lens, large negative, and accurate auto exposure.
My one and only complaint about the GA series is the fact that the lenses are relatively slow. If I could open it up to say f/3.5 I would consider it perfect.
It is easily my go to travel camera.