Altitude is exponential. About 3000 feet is where you might start noticing the effects, and gaining (small) increases in red blood cells. For every 1000 feet above that, the effects increase exponentially. The difference between 5000 feet and 0 feet, is the same as the different between 5000 feet and 7500.
Think of it like swimming to to bottom of a deep pool, but in reverse. The higher you go, the lower the air pressure, and as you conine to go higher, there is less and less air above you pushing down on the air you are brrathing, making it harder and harder for your lungs to fill.
Your lungs full by (basically) creating a low pressure zone inside you that the air rushes into to equalize. The lower the air pressure where you are at, the less is needed to equalize the low pressure zone in your lungs. Hence less oxygen delivery.