Dazed & Confused has it right. To me, I don't see any reason why the big, SEC schools would want to come to this meet. If you can have a high quality meet like the LSU Invitational appears to have, why would you travel hundreds of miles north to a poorly run meet?
I ran at an east coast D1 school, and we went to Penn Relays every year. Never once did I leave that meet saying to myself "man, I'm so glad we came to this."
The history and uniqueness of the meet are great, but that only gets you so far once you been there for the second time. It's no secret about the paddock areas anymore: they're terrible and make feeling prepared for a race extremely difficult. The 8 million heats of everything and the HS races and open races are all great, and they are part of the history of the meet, but it makes it very difficult on the athletes. You're shuffled through the paddocks outside the stadium, and the officials make it seem as though something terrible will happen if you're not inside the stadium in the next 30 seconds, only to then stand in the paddock inside the stadium for like 20 minutes. The meet has a time schedule, but good luck using that schedule to determine when you should warm up. Franklin Field is also very old, and poorly set up to handle the massive crowds. If you run a race at 3pm and want to see any of the races that day, good luck. Better start your exit from the stadium at noon.
And yes the officials treat everyone like crap. But you're in Philadelphia so what did you expect.