that is a very good question.
Some will tell you no, I'm sure.
I am not sure if this has been proven either way, but my personal experience is that it helps me:
1) it helps break up the monotony of just doing miles - I feel better after my temop and look forward to getting back to some mileage after.
2) I think it helps set the stage for future tempos by allowing you do run without any pressure to preform/no upcoming races = so you are more likely to do the tempos right (this is not too fast).
Two summers ago I build my mileage up to 200 max (ave 140-160 for much of the summer) with 2-3 tempos per week...never felt stronger that following fall. the tempo went really well, by fall I was going sub 5:15.
If doing tempos in the base phase was or is detrimental to aerobic development then I would have thought I would see it. I did not. Instead I found that I could do way more reps on shorter rest than ever before once I starting doing workouts.
Ultimately the only way to find out is to try it for yourself. I say go for it - you sound like you have a good plan of not trying to kill them....and gradually getting faster. I would add that there will be days when the summer heat or just tiredness will get to you - so don't feel like you have to do every tempo you have scheduled. Sometime I sit one out - others times I go ahead and do them and find I feel much better afterwards - they kind of wake up the body.