I definitely find my easy pace getting faster as I get fitter. This was the case decades ago back in college, and it is true as a masters runner coming back from an injury.
Like others have said, you do have to monitor yourself some and try to be honest about why your pace is picking up. I tend to find myself dropping down faster and faster out of a combination of boredom (because I run alone a lot) and out of a desire to just have the run be over (since I am marathon training and most of my runs tend to be 10 or more miles).
When alone, I tend to check myself by forcing myself to recite the pledge of allegiance. If I can't get it out smoothly and without changing my breathing, I know I am starting to crank down below easy distance pace.
Conversely, I never worry about my easy distance pace being too slow. If it is slow one day, I just attribute it to being where my body is.
I generally see my average pace for a 10 mile run range over a distance of 20 to 30 seconds per mile on a day by day basis.