Each has its pros and cons: A cave would be pretty cold in the winter, but that would be good in the summer. If hobbits and cavemen could do it, so can you. I’m not sure a company would approve your proof of residence though. How about a tent? Many people do it, you don’t have to pay for a house of property, you just can’t do much in a tent though. In house it’s a quiet life, you can do what you want on your own property.
A house is probably the best choice, but I would like to live in a cave for a while.