I just clicked on this thread. My first job out of school involved a lot of work with electricty deregulation.
You'll laugh when you see the author of this paper:
https://www.academia.edu/6170867/Lessons_from_the_First_Year_of_Competition_in_the_California_Electricity_Markets
I'm surprised 18 years later they're having to pay someone else to take the power. But back when that paper was written, you were paying people a ton of money to be able to ramp up and ramp down, so paying someone to take your power is just another form of this.
Also I'm a bit surprised that there aren't unused resources that don't ramp up when power is free.
And instead of paying someone else to take your power, could you sort of just dump it in the ground? I'm far from an engineer but I guess this isn't feasible? Or is it just not seen as a net benefit for society to just burn off the electricty and waste it. Now they are in essence paying Arizona to act as their ramping agent to keep everything in balance. So net/net Arizonans are getting their power cheaper during these hours, Californians are having to pay a little something for Arizona to take it, so is that the same as just if they could dump it somewhere in the ground?
I'll need to read that paper again but my takeaway on that first year of the deregulated California power markets was that that humans design markets with inefficiencies and utilities were great at exploiting them.