My experience is that the 500PF has somewhat better overall contrast, a small sharpness improvement (mostly visible at 100% crop as you said), and the 100-400 has a
tiny bit of CA with the TC, if you shoot with branches against a bright sky. All that little stuff combined with slightly less reach makes me reach for the 500PF if I'm shooting far/small stuff.
I won't say the 100-400 is anything but great, though. It's extremely versatile, and it's great in the hand thanks to its rear balance and mode ring (which I use for EC).
The 100-400--while large--is a great landscape lens. I've also done a bit of soccer with it, and it did very well. It's not a 400/2.8, but a 400/2.8 also isn't a 200/4.5.