That $500k/year is 'only' $300k after tax (interest is taxed as income). The $10MM house will cost you minimum $100k/year to own. $200k/year after tax not including housing expenses is not exactly chump change, but if you have a family and have to buy your own health care, and want to travel a bunch because you're retired, then you'd have to watch your expenses. Notably, the real (inflation adjusted) value of your $10mm in investments will decline every year, and if interest rates drop, so will your income. With 50+yrs left to go it doesn't sound that comfortable to me. Maybe better to do a $2mm house and live off $18mm of interest income, and reinvest some.
Anyhow, I'm 45 and have a net worth substantially more than $20mm, but I do continue working. I ask myself every day why I do it as I've been doing 'this' for a long time and no longer enjoy it. The only answer I can come up with is risk management and lack of something different I really want to do. There are, of course, many things I'd rather do than work, but most of them aren't practical when you have kids in school.