The US currently imprisons 2.2 million people with direct costs of around $100 billion a year. This doesn't count the cost of removing someone from the workforce and likely impairing any future productivity because of the effect imprisonment has on future housing and employment prospects. Prisoners with children often end up with their kids in foster care or with a relative and the kids tend to be neglected and repeat cycles of imprisonment for petty crime when they are older.
There are plainly people who need to be in prison for the protection of society, but imprisonment also imposes large costs on society. Before someone is imprisoned, there should be a net benefit to society. That's not easy to judge in some cases, but I can't imagine the costs to Japan of imprisoning a former marathon runner for a year could remotely balance out $23 in stolen snacks. The cost to Japanese taxpayers has already greatly exceeded $23, and the threat of prison probably has minimal deterrent effect on compulsive behavior by bulimics who consider shoplifting in the future.