This happens all the time. Usually it is because I just hesitate. Photographing on the street and being scared someone is going to punch me in the face.
I do think though that our minds make the pictures much better than they really would be, and only one self can see them. Not that bad either.