I'm a life-long low mileage guy (20-30), now late 40s, quite thin, pretty well-defined, I hope. First noticed in college that unrestricted eating plus really low miles equaled noticeable softness.
But at my normal mileage, first noticed that I couldn't eat whatever I wanted sometime in my mid-20's. Really pissed me off. And got worse. Had to eat less and less to maintain. Fortunately, it seemed to level off roundabout my early 40's, and I also simply got much more used to the idea that if I wanted to look thin and fit, I had to eat quite a bit less than I'd like to. Oh well.
It's funny (AND really stupid) when people say to me (and other thin folks), "You can eat whatever you want!" And I think, "Really, you think that I only WANT 2 or 3 pieces of pizza? No, I want, and could easily eat, 5 or 6 or 7....I just don't." Dumb.