So, I'll add my story.
In high school, I played soccer in the fall and tennis in the spring. I was one of those dudes who played a lot of sports growing up, and was generally athletic enough to be fairly good at a lot of different ones. Never thought once to join track or cross country, although I ran a 6:58 mile as a 6th grade during one of those Presidential challenges, to which the gym coach thought I could be a good mid guy.
Fast forward to college, nerd school in the East. Join a fraternity, spend most of my time playing beer pong and studying. A couple of my fraternity brothers are on the track team, and are hosting an intramural track meet for the campus to compete. Having played a few different intramural sports with the fraternity, a few of them commented that I was quick and should join the team, as it is typically weak in the sprints.
I show up the intramural meet, and win the 100, 200, 400, and LJ. The only time I remember is the 400, which was 53.3, in flats. The coach is there, tells me that I should join the team when training starts back up in the fall.
I eventually do, mostly to hang out with my friends and pad my resume. Two years later, I hold five school records and are the first All-American sprinter in my school's history. I also continue partying at the same rate, eat like shit, and never lift a single weight. I don't say that to brag, I say that in regret. I took every practice and every race extremely seriously, but had no discipline off the track.
After my final race, I'm burned out. I graduate with a CS degree from the aforementioned nerd school, and I'm eager to start my career and finally get out to Silicon Valley. Years pass, and I never run seriously again.
I regret tremendously because in two years, I went from a complete novice to an All-American, all while doing all of the wrong things off the track, and frankly, as a sprinter being coached by a distance specialist. Do I wonder what could have happened if I had done a few things differently? Yes.
So in regards to your situation, I completely understand that guilt and have in fact started to run again with the idea of maybe running some unattached meets just for kicks. I wish I hadn't stopped all those years back though, and I suspect you'd feel the same.