The main problem with vacation properties is that you end up being married to them and are reluctant to go anywhere else. The other problem is that whenever you go out to your vacation property, you spend half your time cleaning up and fixing stuff. And you will have the same problems with neighbors, roads, utilities that you have at home. A friend of mine bought a great property on a river in Central TX near Austin and San Antonio. He actually makes good money on VRBO-ing it. But the ranch on the other side of the river has a half dozen dogs that are kept chained up in a yard and bark for 8 hours a day. Whenever anyone has gone over to complain, the owner comes to the door with a shotgun. The police won't do anything about it because they know that if they show up, they are going to get into a firefight with the guy because he is a militia wannabe and has tried picking fights with police before.
Vacation homes can be great investments and they can also be a dud. Lake houses up north of Houston used to be cheap. People would buy lots for $10-20k and build a little lake house for $50-60k. Now, the lots are worth $300-400k and people are building 4-5,000 sq ft houses that sell for $1 mil+. But my dad bought an old farmhouse in the middle of nowhere in Vermont back in the 1990s. He put a lot of money into renovating and maintenance over the years. He would spend summer there and winter in FL. He never got any interest in renting it in the winter because it is at least 90 min from any ski area. And he is now trying to sell it and hasn't got any offers. Location still matters in a big way with vacation properties. You can find something cheap in the hills in the middle of nowhere, down the street from a guy living in a trailer with a thriving meth lab, but that doesn't mean that someone else will want to buy it when you are done with it.