I didn't say it isn't a factor, it is. But based on the history of temps and CO2 levels, it isn't a controlling factor. For CO2 to be as crucial as Al Gore and IPCC would have us believe two things have to be true:
1. The greenhouse effect drives the climate.
2. CO2 is the most influential greenhouse gas.
We already know 2. isn't true, CO2 only accounts for 3%-20% of the greenhouse effect depending on the study. (Science has yet to pin down something as simple as that.) The effect of water vapor is many times greater than CO2. Large magnitude changes in CO2 haven't been shown to cause large changes in the greenhouse effect.
The first point is also not true. The greenhouse effect is a buffer. It keeps global temps operating in a narrow range range. But the greenhouse effect can't by itself cause anything to heat or cool, there has to be an outside energy source. And many other factors determine how much of the outside energy reaches the earth and how it is distributed over the earth's surface. These other factors have as much, and often much more, influence as the greenhouse effect.
Human CO2 emissions account for about 3.5% of annual CO2 emissions. If we assume CO2 accounts for 20% of the greenhouse effect, then we cause 0.7%. If the greenhouse effect is the ONLY reason for warming, then we're causing 0.7% of it. Yes CO2 can contribute to warming, but our contribution is insignificant. Eliminating it won't mean squat.
"Just because other non-CO2 things caused the Earth to warm in the past, it doesn't force us to conclude that a non-CO2 thing is the cause this time."
You're right it doesn't force us to conclude that, but it would certainly make sense. Especially since the current warming started about 150 years before the Industrial Revolution.