For me anyway, it's something I don't know a great deal about and have had little to no exposure to, and also is something that's perceived as for rich people due to the locations of the events and how accessible they are.
For example, in high school in terms of running, we all tried 60m, 100m, 200m, 300m, 400m, 800m, 1500m and there was the option to do 3000m. And then there were local fun runs (2 miles), XC (about 5K). There was never anything longer. So you get used to running shorter things. Also there are age restrictions which are a lot stricter here (13 & 14 - 6k
15 - 10k
16 - 16.1k
17 - 25k
18 & 19 - marathon).
There are no ultras near me, apart from one that is 145 miles that looks insane.
It just doesn't seem like running, it seems like a different sport. For example it says that "Competitors may enter vehicles for warmth, rest and nourishment, but will be disqualified if the vehicle moves". I can't imagine getting into a vehicle during an 800m! So then it feels like maybe it has more in common with hiking or something similar.
So I guess it feels like it's outside of my frame of reference. A bit like these adventure races where you get in a canoe for part of it.
That doesn't mean I'd look down on an ultra runner, I just see it as a different sport in a way. Just like I see sprinting as very different to fell running.
I also think the confusion/non standardisation of the times doesn't help, it's very course specific. Runners are absolutely obsessed with time and PBs and the thought of the time not meaning anything is foreign. So one time on one course could be slower but actually a better performance than a faster time on another course. So unless you have insider knowledge you're not going to relate to that.
There is also the view that if any 'serious runner' tried an ultra they would smash all of the records. I don't think this is true as it has a different skillset. But maybe some runners view it as not competitive and get annoyed about that, which is a bit daft. I'd love to see a competitive marathon runner do a track ultra though. It would be interesting to see if their talent for the marathon translated into e.g. 50k, 50 miles etc.