There is no good answer here. Frosh and Soph times don't matter much but they can contribute to the trends that coaches will be looking at when they are recruiting athletes as juniors and seniors.
For example, my very successful local HS team has two types of kids:
1. A group of experienced age group runners that have followed a well worn path to this school. They've run regularly since middle scoolers.
2. General students that have joined the XC team with no experience.
Right now there are two juniors in the top-5, one out each of these two groups, and if I was a college coach I'd take the kid out of group 2 since he has more upside and has reached the same level on 2 years of serious running. A coach would look at his freshman times and realize that he had no prior experience.
Two of the seniors at this school, are group 1 kids and, while they have run fast, you can look back and see that their improvement curve has been very shallow.
The most insightful answer I've heard is that college coaches will know who they are going to recruit by the end of that classes' junior year. Senior year is mostly about taking final visits and sealing the deal but coaches won't be adding too many names to the list after your junior year of track.