I agree. My friends who are professors work very few hours in comparison to my D1 coach friends/family. They literally make it to every single one of their kids' competitions or school performances.
D1 (and DII) coaches work 6-7 days/week, including overnight travel. There are no "summers off," either. Field event coaches work one-to-one in technical instruction, on the track, 15-30 hours per week. Their "Office" is their smart phone, in their pocket. Administrative work, travel, eligibility, recruiting, university meetings, etc. are beyond that. Planning and running summer camps to add to salaries are a complete hassle, too. Coaches with elites have to do extra work and travel - usually international travel all summer - beyond their university obligations, and that work is sometimes/often volunteer. Competitions are not 3 hours, like soccer or basketball, they are (as we all well know) two 12-hour days, more days for championships.
How many professors are involved in your life personally as an undergrad? Profs lecture at 1 hour classes... a full load is up to 8 courses, which would be a maximum of 24 hours lecturing. Of course graduate students often TA undergrad courses, especially for freshmen and sophomores. Four classes are more common, which is 12 hours/week lecturing. Obviously grading, office hours, and curriculum preparation are outside those hours, and research is above and beyond that. For tenured jobs, "Publish or perish" is the rule.
From UC Berkeley:
"At a major research university or top-ranked small college, the teaching load is typically 2-2 (two courses per semester, and at a university you may teach graduate and undergraduate versions of the same course each semester) in the social sciences and humanities - less in the sciences and engineering. At the other end of the spectrum, there are many colleges and some universities where faculty carry a 4-4 teaching load. Even in the latter case, it is unlikely that you would be asked to teach eight different courses, and a distinction is commonly made between the number of courses you teach and the number of preparations (i.e., teaching the same syllabus more than once in a semester or year). The number of preparations you are required to teach may be almost as important as the number of courses, and this is often negotiable for first year faculty if you remember to ask."
Now, head coaches do make a disproportionately inflated salary compared with their assistant coaches, who typically coach far more athletes, but that's a separate discussion.