Do you generally use a yellow or orange filter when taking street candids outside, or is the loss of a stop or two not worth it? I personally find white skies distracting and always enjoy a bit of dynamic between the sky and clouds. I find that most shots are taken without a filter, however I occasionally come across a few that are just so dynamic thanks to the sky.
A yellow filter will correct the spectral sensitivity of B&W film a bit closer to that of our eyes. It won't do much for for blue skies and nothing for bright skies. If you suffer from harsh contrasts in your street photography, you may be interested in trying XP2. It has a high sensitivity and can deal with contrast like no other film I know.