When I played around with street photography a long time ago I would try to have the camera already focused for distance and then walk slowly down the street. When something interested me I would then stop and point the camera in another direction and then quickly turn and catch the subject and then rotate back to what I pretended to be looking at.
Since I used a motordrive, the subject may hear it and look up but by then I had rotated back in another direction and they weren't aware that they had been photographed.
I also practiced shooting with both eyes open so I was never confined to the viewfinder. That way when looking through the viewfinder in the fake direction, I could still see my real subject and quickly pivot back to capture them.