I've seen this discussion on some other camera forums and sometimes it gets ugly. Hope nothing like that happens here. My point of view is it is a personal choice. I use B+W filters over my lenses. I have one lens that has a Sigma filter (I think one of their WR) that came as a promo with a Sigma 100-400 lens.
I have scratched a couple filters over the years and had one break from an impact. As a wildlife and nature photographer, my lenses and camera bodies are out in some pretty rough stuff.
It may hurt IQ some but, honestly, I've never seen any difference. It is awfully hard to test IQ with same lens same scene under same exact light filter on and filter off.
With my Nikon 105mm Micro lens, I shot it for about 2 weeks without a filter as I was determining if I wanted to keep the lens or exchange it. The lens was a keeper so I got a B+W filter. Looking at the photos, I don't see any difference other than later images are better but that had more to do with me learning the strengths and weaknesses of this lens and more importantly the strengths and weaknesses (mostly weaknesses) of my technique. I don't see where the filter made any difference one way or the other.
I know it's not a "definitive" answer but with good filters (yes they can be pricy) the difference, if any, is so minimal you'll never notice it. If you get flare shooting directly into a backlight situation, take it off but I haven't had that problem either.
I hope this discussion was helpful.