Rights play both ways: you may have a right to photograph in public, but people have a right not to be photographed in way that violates their personal space.
Taking casual photos of people in general on the street is probably OK; sitting across from someone and taking (or attempting) to take several photos of their child at close range - without talking to the person first and asking permission - is in bad taste. If someone I didn't know was attempting to take photos of me under similar circumstances, I'd tell the photographer to F-off!
Absolutely correct. This is where respect and human decency becomes somewhat conflicted with the letter of the law.