Well, if you look back to the last time the Earth experienced a temperature change, you'd be looking at the era between the 14th and 16th centuries called the Little Ice Age. Now the average temp only droppeed 2-3 degreed C but....that small drop created global freak weather patterns. Snow in July, the year without a summer, etc....
Now, while we won't necessarily see the same types of weather. I am sure we'll see the other end of the scales. Droughts become way more intense, storms are bigger and stronger, heavier monsoons, more flooding. And while a year or two without a winter sounds like a good idea, I know for the more northern regions winters kill off a good part of the bug populations. The number of cases of bug borne diseases in both humans and wildlife will skyrocket, and our lives will generally be more miserable as a result of them.
While it might be considered better for agriculture, an increase in the bug populations means more pesticide use, and like we saw in Iowa and along the Mississippi with the flooding this spring, freak weather patterns can destroy our food crops just as throughly as cold did in the 14th century.
Whether or not people want to believe humans are responsible for the warming of the earth, or that it is a natural thing, or that it is all a bunch of smoke. The geological evidence is there that humans could be facing some really really tough times if we don't adapt to our new and changing environment.