Academics, for the most, are what you make them. The name on the diploma is not as important as what it used to be. Hiring managers may give preference to Universities when they recognize the name, but mostly won't differentiate between small, private liberal arts schools and directional state Universities.
If you need more of a challenge, take on more credits, graduate early and start a master's during your eligibility. Upper level academics are not necessarily all the difficult , where ever you attend.
In some senses it is best to be big fish in a small pond. Being well known and liked by instructors, winning department wide academic awards, doing research...these things can lead to jobs and connections on the outside. From what I have seen over the years, the 36 ACT student who goes to a mid major U and graduates with top honors opens more doors for themselves than the 36 ACT student who graduates middle-third of their University class at a top school.