You are 100% on point with this.
Worldwide we have 530'000 people infected and 24'000 deaths - a 4.5% death rate which of course includes people of an advanced age, health complications, people who smoke etc and as we now know some healthy people unfortunate enough to have bad initial to responses to infection (chaotic interferon overload) which have lead to complications. The US has 84000 active cases with around 1000 official deaths.
In the US alone unemployment jumped from 215k to 3.3 million in ONE MONTH. Many people championing (and I use as a sample the people I know) the current courses of action are of course in very privileged positions of not having to worry about this (salaried full time employment).
Put it this way, if COVID-19 was a completely asymptomatic virus what would we do (btw the world is full of asymptomatic viruses that we all carry), what would we do? Answer would be nothing - we wouldn't need to. So we are, as you said, doing this solely to protect at-risk people. But at what point do those people simply take it on themselves with respect to extra precaution and common sense? Are we okay shutting down society and okay with so many people in our society losing jobs and digging themselves into long-term holes just because of this? This can't be the answer, it simply can't.
I wonder then since we are making such enforced sacrifice for at risk people (which include for example smokers), do we get to enforce sacrifice for them once this is over? Next time I see someone plowing through a box of Marlboroughs do I get to enforce a sacrifice by grabbing the cigarettes and tossing them? Explain how that wouldn't be fair and reasonable?
It probably wouldn't be seen as acceptable and to be honest neither are the overtly extreme measures we are taking that as you said are trashing our economy and literally ruining the lives of considerably more Americans than COVID-19 will. That is not even up for debate.