The stock response should be, "I'm sorry, officer, is there something wrong?" The next question should be, "Do you think I'm breaking the law?" If the answer is anything other than the affirmative, the next response would be, "Excuse me, officer, but I'm busy and I must be going, now."
Simarlarly, in the UK, the suggestion is to ask an officer "are you detaining me?" If the answer is "no", you can move on. Luckily we don't have to worry about private citizens or cops with guns*.