Crankset length has long been a debated topic in cycling (it has been for the two decades I’ve been interested in the sport for sure). When I was first getting into cycling the big push was for longer cranks. My 54cm bike came with 175s. I eventually settled on 172.5 because, while I liked the torque offered by the longer cranks, the extra knee extension required to pedal them gave me issues.
You mention the knee/hip motion but you ignore the fact that a shorter crank has drawbacks, too, namely lower max torque and shorter length of power stroke. Those aren’t minor concerns either. So it’s obviously not just shorter is better. There’s a balance and it will depend on the discipline with track cycling, triathlon, time trialing, criteriums, and hill climbing all being different.
Super high cadence is really only relevant for track cycling (single speed bikes). Extreme aero positioning and the bike fit changes to accommodate it are irrelevant for hill climbing. And what pros can benefit from and what the rest of us can live with are often worlds apart.