The hardest race I've ever run was also amongst the slowest, when I was trying to come back to form. It hurt more than anything else. It was an 800m, and on the 2nd lap I ran way beyond my current fitness. I was destroyed for an hour or so afterwards, felt seriously unwell. I've run much faster and almost coasted it. So to me, that answers your question. It's not necessarily about the speed itself, but the speed relative to your fitness and often, when you're in good form, you're more relaxed and it simply doesn't hurt as much.