You're right. If you ask Jeff Galloway, Amby Burfoot, or Bill Rodgers, they will all tell you that their Wesleyan education was second-rate and they wish they would have gone to a larger university.
And I have plenty of friends who try to get jobs in major companies and once their prospective employer finds out they went to a NESCAC school, they simply invite their colleagues in the room and gather 'round for a big laugh. This is especially true at companies like Proctor & Gamble, AOL, Morgan Stanley, Colgate-Palmolive, or even the Yankees, because all of the CEO/President's of these companies went to NESCAC Schools.
Keep drinking the kool-aid.