The reason the power 5 conferences are the power 5 is budgets.
https://sports.usatoday.com/ncaa/finances
According to this, the bottom power 5 athletics budget is Washington State. The only non-power 5 school la that have a larger budget than Washington State are UConn and Houston. Granted, this list does not include most private schools, so I am sure there are more. But there is a big gap. The American is competitively very good, but there still is some gap, and they also have more of their budgets funded through student fees. A power 5 school may have student fees, but they aren’t accounting for half of the total athletics department budget like they do at schools in other conferences. That to me is the difference: Power 5 athletics departments have revenues (through ticket sales, TV contracts, etc) that enable them to be mostly self-sufficient.