I have had photos with weird reflections from the filter. I didn't realize how often it happened to me until I went back and looked at hundreds of old pictures. It was over 20%.
I mostly take outdoor photos in sunny/daylight conditions. But I also had a lot of flash photos where it happened.
I also learned that I don't like polarizers either because they kill sparkly highlights like what you get on water.
I seldom use a filter now but I try to always use a lens hood.