Sure, many people unsuccessfully dabble in the plant-based lifestyle and end up returning to eating meat. But think about how many other things people quit at. People quit running, quit the ketogenic diet, quit learning a musical instrument, and generally fail at sustaining most goals.
Yes, plant-based nutrition does require a bit more thinking. If you take the standard American diet and simply remove the meat, you will become even more nutrient-deficient than you already were. So some people quit veganism simply due to lack of proper planning. But the Academy of Nutrition and Dietetics has many times explicitly stated that well-planned vegetarian and vegan diets are healthy for people of all ages.
Some genetic factors do predispose some to success or failure on plant-based diets. For instance, conversion of ALA to DHA and EPA long-chain Omega-3 fatty acids differs from person to person and even varies based on gender. So some vegans get plenty of long-chain Omega-3s just from things like chia and flaxseeds, while others end up deficient. Plant protein is generally slightly less bioavailable, but differences are marginal. For instance, protein in foods like nuts, oatmeal, and beans is only 5-15% less available than animal protein, so the simple solution is increase your protein by 5-15%.