It's typical in your 40s and 50s to look back on your life and second guess things you can't change with a big game of "what if." Be aware that it'll rob you of the joy of the great things you already have in life.
Invest in your wife and invest in your kids. It will pay rich dividends. Don't do it just for them to reciprocate. Humbly do it because they deserve it. As a guy who's kids are long grown, the greatest thing in the world is the love of your wife and the love of your kids.
Commit yourself to make your wife the happiest lady in your neighborhood. And your kids. Go out of your way with your time, being pro-active, take the initiative in creative, spontaneous, and amazing ways.
If you could be a fly on the wall of the homes of that old affluent neighborhood in which you used to live, you'd discover that many of those houses are full of things but empty of love. The family members tolerate each other, and don't thrive on each other. Life is too short to live that way.
Your kids are still young (can't be older than 11) and you and your wife are still young at 41. Nine times out of 10, if a man takes the initiative to love and cherish and adore his wife, she will respond in very very positive ways.
And FWIW, spending all that money to have kids of your own? Worth. Every. Penny. Your wife, kids and grandkids should be the greatest things in the world, but it will only happen if you invest in them with love.