You can’t blame the strangers you are photographing when they confront you. No matter how much you know your rights and the law, it doesn’t matter. Change the game you play and learn the unspoken rules of the streets. Your confidence will grow with time once you learn how to dance. A lot of practice comes down to having proper body language.