"On a course that has been run by large numbers of elite runners over many years (e.g. McAlpine, Mt. SAC, Van Cortland Park) it is reasonable to compare times."
This is NOT true. Some years are muddy messes, some years the grass is dead and the ground is hard and fast. Generally, the older a course gets the faster it gets. Each year they widen turns allowing shorter tangents. Each year they smooth out rough sections and the ground gets more packed down and supports less energy absorbing grass.
The course record of the big course I ran on in HS has improved by 60 seconds in 35 years. Both records were run by guys who ran 9:20 for the 3200 as juniors and who never broke 9:00 in their lives. I'm sure the average runners are 30 seconds better but the other 30 seconds is course changes.
We compare performances run on tracks even though we know the surfaces of those tracks are different and the weather conditions may have been better in some places than others, so I don't see a problem there. Those who run in poor conditions just got unlucky.
With regard to old courses getting faster over time due to soil compaction... that's an interesting theory, but we haven't seen that in Illinois. The high school State meet has been run on the same course since 1971, and the evidence suggests that due diligence has been put into maintaining the course distance over all those years. Craig Virgin was the first State champion on this course and set the course record which has not been broken since. We have had Olympians and other notable runners such as Jorge Torres, Don Sage, Lukas Verzbicas, and Chris Derrick all come within striking distance of the record. If the course were 30 seconds faster than in 1971, then one of these guys would have beaten the record. There is no evidence that the course is getting naturally faster over time.
My guess is that your theory on wider turns and other course modifications shortening the course is correct. If you don't have reliable information for the historical length of the course, then the length is suspect and comparing times over many years will be difficult.