late Nov is hit or miss on the weather in north FL. It wont be cold...it IS florida after all...but it won't still be very warm either. Gray skies and mid 50s is not exactly beach weather. But the water will not yet be too cold to swim. That takes longer, because the gulf stream is basically just right off shore here.
Anyway, Jax beach is pretty good. But if you're headed down to St. Augustine, just go to St. Aug beach, or a little further south on A1A to Crescent Beach if you want it a little quieter. You can drive onto the beaches there.
At the north end of st Aug beach is Anastasia State Park. Good beach and good amenities (bathroom, showers, grills and picnic tables) if you're interested in having that around.
Historic St. Aug is fun, and if you can sift through the touristy stuff, there is some good history to be learned. Castillo de San Marcos is nice, but not wonderful. The St Aug lighthouse is very pretty and offers a great view from the top. And walking around the campus of Flagler College is cool - it was an old railroad hotel. Beautiful architechture. And then, of course, we cannot forget the Alligator Farm. If you've never seen a 15 footer up close at feeding time, go see it.
Places to eat while there - The Columbia is very good latin american food, good steaks, nice place, 20-25 a plate, and right on St. george street. The Oasis out in st aug beach (across the intracoastal from st aug proper) is a very well known local joint with great burgers and sandwiches, many taps, live music etc. I highly recommend the Whaler sandwich. You'll see it on the menu and think "porn stache is crazy" but trust me! A1A ale house in historic st aug has very good beer. For Breakfast, find Greenstreet's. Order the Skillet Mess, and thank me later. soooooooooo goood. Or go back to the Oasis. They serve a good breakfast too.