You are no longer part of the tribe so you are suspicious and need to be closely observed to ensure that you are not a danger.

tkamiya is right, buy a camera phone. When someone questions you, whip it out and act like you are taking their picture. Suddenly you are no longer interesting and they will go away.