Just because you might not want to workout with rivals doesn't mean you aren't passionate about the sport or anything else negative, I don't care what Malmo or anyone else tells you. Everyone is different, we all have our different niches.
With that said, if I had never worked out with "rivals" I would have far less memories and friends involved with our sport.
Some of my fondest memories involve cool down runs with a great competitor, or Sunday long runs after a race with rivals, before they headed back home.
Doing speed work with a rival? Nah, not for me, but that's just me...
I imagine post collegiality things would be different, that would be fun I'd imagine, I certainly wasn't good enough to ever be in the position of meeting up with a fellow professional to run some 400's.