Yes, your undergraduate institution matters to the admissions officers. Look at any first year yearbooks of top tier law schools and you'll see many, many Ivy League kids (or other top schools like Duke, Stanford, U of Chicago, Georgetown, Northwestern, or competitive liberal arts colleges).
I agree with the prior poster who said that if you went to a big state school and get in, it's likely because your LSAT score rocks (i.e., 170 or better) (and GPA as well). I think admissions officers look to the state school students to boost the average LSAT and GPA, and give other kids a break if they went to an impressive undergrad.
If your LSAT and/or GPA aren't that high, don't worry too much. You can still get in to a top tier school through the back door -- by transferring after your first year. I did this (to Michigan), and after your first job it becomes irrelevant how you got there. People only care about where your degree came from. Not all top schools accept transfers, though, and even if they do, you pretty much have to ace your first year.