I received my 180-600 in March and in April got my hands on a 500pf, with the plan to only keep one. I have been shooting both since and trying to make that decision. While the question of zoom vs prime and the different experiences of using them is one that was on my mind initially, in practice I have rarely ever thought about it. When I have the 180-600 on the camera I will zoom when the opportunity arises. When I have the 500pf on there, I either change the field of view by moving or if that's not possible aim for different compositions which work with the field of view as I have it, with the key being that all of this happens naturally. In other words, I never really think about it and say, "I wish I had the zoom right now" - the photos just come as they present themselves.
So in making a decision that is similar to yours the zoom vs. prime has been a factor, but one that is smaller or I could say is in a broader context than I would have thought.
My initial concern with the 180-600 was in terms of IQ, so I have been doing a lot of comparisons IQ wise, above all. As expected, the 500pf is sharper, but not by much. Thus, when considering the zoom the more nuanced question I've been trying to sort through has been whether the ability to zoom on occasion is worth the difference in IQ I have noticed. This also comes down to cropping. The slightly better IQ of the prime stands up to cropping a bit better than the zoom, so if we assume I am shooting the same smaller subject with both, how much better results can I get from a crop with the prime, and is THAT worth the tradeoff?
A second factor which I have been considering is the AF. The AF on the 500 is definitely faster, but in a way it feels less smooth than the 180-600 and sometimes I wonder if it's almost a kind of disadvantage. For instance, recently trying to photograph swallows zooming around the water I had both lenses to compare and I had better luck getting focus with the 180-600. When I was younger we once saw the kind of extraordinarily expensive sports car on the road next to us that you don't normally see out on a regular road. He commented that when you drive one of them you had better make sure your wheel is pointed where you want to go before you hit the pedal because it is going to go there fast when you hit the gas. That's been my experience with the 500pf's faster AF. It's like it's so fast that if it doesn't quite get focus as the focusing element zooms "towards" a subject that it just flies past and keeps going and so it takes longer to come back to the subject. With the swallows, I had the lens' focusing element flying rapidly to bring them into focus while the birds were literally flying rapidly towards the lens and so in practice by the time the bird came to a certain distance from the camera the focus had already flown past that. With the slower 180-600, the slower speed almost felt like an advantage so that in those fractions of a second that the AF system was evaluating things the focus was just slow enough to still be ahead of the bird and be able to get it when it came into the right plane.
I've also been considering aperture. I have seen two kinds of reports about the 180-600: some people find their copy sharp at 6.3, while others have found it a bit soft there with dramatic improvement at 7.1. My copy is the latter. At 6.3 it's often disappointing, while at 7.1 it is much better. This means that I am deciding how much the 2/3 of a stop over the 500's 5.6 matters to me. If your 180-600 is sharp enough at 6.3 then this isn't a factor, but if it needs to be stopped down you may need to think about the difference between 7.1 on the zoom vs. 6.3 on the 600 and whether that matters to you enough that in practice it will make you want to avoid the 180-600.
I have been a bit longwinded here, but I suppose my main point was that at least for me, the question of whether the zoom is worth having around has wound up being much more nuanced than just the question of the value of the zoom. The decision is slightly different in that I am deciding which lens to keep and you seem to be deciding whether to keep both or only the prime, but the fundamental point is similar: the zoom itself hasn't provided as much of an advantage over the prime that it makes me think about it on its own terms when making my decision. It matters, but only in a broader context.
