Hey OP:
That is a very grown up choice you are wrestling with. What do you want? What do you stand to gain or lose? Do you want to be on this team, or do you want to be a normal college student? Which path will help steer you toward the adult you want to be? Will you regret quitting the team without giving it a fair chance? There are certainly pros and cons to both - having picked the team/running route some 22 years ago I can only comment on my experience. Since you chose to post this on a running forum - I'm going to give you some encouragement and insight from my experience. You only get one shot (with a few exceptions) at being a college athlete. If you decide you want it, I encourage you to go for it now, make a commitment to the lifestyle and don't look back. However, if you are not committed to being a college athlete it most likely be a negative experience. Being a successful student athlete takes a big commitment of your time and talent, it takes a lot of hard work. I would stop here but you said one think that I really liked, "you hate the idea of quitting." If you hate quitting that is just a different way of saying that you are willing to work hard and not always take the easy path. That same mentality kept me from quitting xc my freshman year of college, and I'm glad that I stuck with it and encourage you to do the same. That same mentality is what it takes to be good at anything - so challenge yourself, make this team. Don't go live with your best friend from high school, that's the easy way out and won't help you grow. Make new friends, challenge your body and your mind. Learn about people (like your roommate) who are different from you - it will be good for both of you.
The first 6 weeks of freshman xc was very difficult for me - up to that point in my life it was without question the most challenging time I had experienced. It was both a physical challenge transitioning from running flat at sea level to running on hills and at elevations of up to 5,000 feet. At the same time it was mentally challenging getting accustomed to a new coach, competitive mindset, being away from my family/support base, and adjusting to not being one of the top guys (reality check). I thought I was going to show up and be pretty good. I quickly learned I was not where I needed to be fitness wise, but I decided I would hang in there (I got lapped in our VO2 test!). It was hard, I thought about quitting - like you, I had no financial incentive (scholarship) to stay - but after some sole searching I decided I wanted to run, I wanted to get better, and I wanted to be on the team. When I made that decision I never looked back, I went for it and it worked out. It took time, about 6 -8 weeks, for my body to adjust to the training, but then I started to improve, I made friends, and I started to enjoy running again with a new mindset of wanting to be a good runner for my teammates. Eventually my teammates excepted an awkward freshman with bad form and a hitch in his stride. I got lucky - I made really good friends on the team and ended up living with them later, I did not have any major injuries until after xc my senior, and I continued to improve each year. Some 22 years later I rarely talk with any of my high school friends (and I had some pretty good ones) but I talk with an see many of my college running buddies frequently. The hardships, challenges, and successes that we experienced forged friendships and bonds that will last a lifetime.
I think many college runners have felt exactly like you do right now - I know I did some 22 years ago. Other freshmen on your team probably feel the same way. Reading your post brought back a lot of memories and freshman xc is very much a character building experience. I encourage you to "think" about what you want rather than "reacting" to a challenging situation. It is your choice, and you need to do what is right for you - but I like your "I hate quitting attitude" and I think you should give it some more time. Everyone is different and not everyone excels at running in college, but you won't know unless your try, and success is not always measured by a stopwatch.
I want you to know, there is a light at the end of the tunnel, but it entirely depends on you and whether you "want to be there, and want to be part of the team." I think you want to be there, and if I'm right then I encourage you to commit to your team. If you do then they will be there for you, they will help you when you struggle, they will celebrate with you when you excel and you will do the same for them.
Honestly - the fact you are getting smoked by everybody on the team (I know how you feel - it sucks) is a good thing. You want to be a place were you can improve. Running/training with better runners helps you improve - you are in a good place. If you train smart (listen to your coach, don't kill the easy days, don't stay out all night, run the workouts smart) and stay healthy - chances are you will improve, maybe substantially. Then in 22 years you can tell someone who is having a rough patch that it was worth it. Good luck!