Everyone will be different in terms of what they believe is "too wet" to go without some rain protection.
When I first started out shooting motorcycle road racing I bought a hydrophobia cover for the 70-200 f2.8 and 500 f4.
I've still got th covers and they still work well on the Z9 and Z70-200 and Z400f2.8.
Has become habit now, 8 hours plus trackside in pouring rain with a rain cover on it's jut one less thing to worry about.