I had a beach house for about 6 years and it was not worth it. We loved the house but as Roy said, I spent the weekends/weeks fixing stuff and cleaning while the family enjoyed themselves. That is fine, but be prepared. And that wasn't even the biggest issue... The bills were awful. Everything from taxes to utilities feel 4x as bad when it is a vacation home. So I sold it at the bottom of the market and lost money, but I am glad I did.
I was in a high-demand area and understand that you may just want a steel structure in a very remote place with no/limited bills. Okay, but then you need to consider the surrounding area and realize that you may have other issues to deal with. My brother is an avid hunter/fisher and bought land way in the middle of nowhere Appalachia and built a steel building with bunks, a generator for light and heat, and a demising wall for where he kept the outdoorsy stuff. He had a tractor and two 4-wheelers, plus some fishing gear, a small metal boat, and even some shotguns locked up in the steel building. Got that all built and secured and was feeling like a million bucks after his first weekend out there.
He shows up later that month and everything is gone. Calls the police and they say sorry but it was likely stolen and they will see if they can find something out. He gets a visit from a guy down the street - old guy - who tells him it was likely just guys in town and that he will watch the place and keep it secure for $XX a month. So he agrees and nothing has happened since. Been about 20 years.