If you like big cities, New Haven will probably not be enough for you. If you like small cities, I think New Haven is a pretty good place to live and run. We are a 2 hr drive from Boston and 1 hr 45 min train ride to New York, so if you're looking for quality road or track races, you can choose from either of those. Believe me, there are enough open track meets that you could race on the track 40 out of 52 weeks of of the year if you wanted (just a guess). The running around New Haven is pretty good too, we have a good series of trails within a 15 minute driving radius. The Regional Water Authority maintains them nicely.
Vibrant restaurant scene. The bar scene exists, but I've kind of gotten over it, there are maybe 10-20 bars?
You hear about the crime and it's much ado about nothing in my opinion. It's a city, yes there is crime. However I do not feel unsafe, and I run by myself at night all the time. OK, I probably wouldn't want my wife to run alone at night, but that's true of any city.
There are grad students and young professionals, definitely. However, it's not as extensive a scene as you're likely to find in a bigger city.
If you're willing to drive a little, the New Haven suburbs can be very nice. Branford, Guilford, Madison, Milford, Woodbridge, Orange. Those are all shoreline towns, so there is a little bit of wealth, so the public schools are in good shape. Lots of AP classes, extracurriculars, the expectation is that most of the graduating class will go to college, that sort of thing. The one downside is that the suburbs are kind of homogenous, as suburbs tend to be. White middle class boringness. Less so in East Haven, West Haven, New Haven, but definitely going further up and down the shoreline, or even into the valley.
I do like New Haven. Sometimes I get bored of it, but other times I really like it. If I were to raise a family I would probably do it just outside the city so I could have a yard and the schools are a little better.