Yashica Electro 35s are dirt cheap, very light, good, and have a fast lens. They are not "compact", though. They take any battery that fits the hole and is 6V or thereabout. You have to devise some way to fill up the air space that was originally taken up by the original battery, which is larger than the ones you get today. I just wrap the battery in tape to take up the remaining width, and use a short, sawn-off piece of steel rod to take up the remaining length.
They cost almost nothing, weight almost nothing and are almost silent.
The drawbacks are: 1. fixed lens, and 2. auto exposure