Don't worry about it. XC, indoor, outdoor -- it's a long season with maybe a week off in between. You need some recovery time (physical and psychological) before you dive into summer mileage. School's out, track is over -- just relax and recharge.
Also, you've peaked for outdoor season. You can't expect to maintain that level of speed permanently. It's called "peaking" for a reason.
For HS athletes, Coach Jay Johnson recommends one week of doing nothing, plus two weeks of "active rest" -- no running, but tennis, biking, swimming, etc. Just relax and do easy stuff. Nothing hard. That's a little too much for my taste. I just do nothing for two weeks in order to unwind, get rid of any little aches I might have, and so on. After two weeks, I start itching to get back out there, so I know rest time is over.