Hands down, it's Buffalo, NY as both city and a downtown. Even downtown Detroit is more appealing (historically and architecturally-wise)
I spent probably 30 weeks over 2 years on a project in this city, but for a "downtown" in a "city" the absolute saddest has to be Virginia Beach, VA.
Their downtown consists of about 10 buildings that are all pre-fab construction and were "created" about 10 years ago, when the city leaders said "We are the largest city in Virginia. We need a downtown." They also have the "tallest" building in Virginia, which is a hotel/condo building that looks god-awful ridiculous, given there are no other tall buildings around it. Again it was a "We are the largest city in Virginia, we should have the tallest building!" mentality. No mass transit. Not very walkable, at all. Traffic is atrocious in the area of their "Downtown" (which they actually call Town Center).