Having shot wildlife for over 40 years using SLR, DSLR and now Mirrorless my take on it is your movement, scent, and noises you make are far more likely to spook wildlife than the shutter click. Just the act of bringing a camera to your eye will spook a lot of wildlife including birds. Can the shutter sound spook wildlife? Under the right conditions sure. I'm only saying our presence in the wild with them is more likely to spook them. A car door closing, a sudden movement, stepping on a stick and causing a snap sound, for mammals, our scent because wind is at our back etc are all likely to spook critters. Over eons they have evolved to be weary of sounds predators or potential predators make. Anything we do that makes us look like a predator or sound like a predator is going to spook them.
With all that said, I do like the quietness of mirrorless but over 40 years of being in the field with my cameras, I bet I could count on both hands the number of times the sound of my camera shutter actually cost me a great shot.
Hope this helps. It can be a controversial topic and everyone has their own opinions and personal experiences. Yours may differ greatly from mine.