Choose where you'll be most happy at. Hard to turn down Stanford though IMO
Re: academic difficulty, i think youll be fine wherever you go. Year in and year out they turn down lots and lots of applicants all very capable of doing the work but they dont have room for. So they pick only those applicants who offer special talent to better the institution, whether that be athletics (you), music, diversity, etc or legacy (not really talent but helps the school for money/reputation)
I took Calculus at an Ivy over the summer when i was in high school (with mostly matriculated pre-med ivy students) and was able to get through it with an A and my SAT was only in the 1200s