Physical exercise and dieting are well-known and effective methods for fat loss and improving cardiovascular health. However, different individuals often react differently to the same exercise regimen or dietary plan. While specific individuals may undergo substantial fat loss, others may observe only limited effects. A wide range of inter-individual variability in weight gain and changes in body composition induced by physical exercises and diets led to an investigation into the genetic factors that may contribute to the individual variations in such responses. This systematic review aimed at identifying the genetic markers associated with fat loss resulting from diet or exercise. A search of the current literature was performed using the PubMed database. Forty-seven articles met the inclusion criteria when assessing genetic markers associated with weight loss efficiency in response to different types of exercises and diets. Overall, we identified 30 genetic markers of fat-loss efficiency in response to different kinds of diets and 24 in response to exercise. Most studies (n = 46) used the candidate gene approach. We should aspire to the customized selection of exercise and dietary plans for each individual to prevent and treat obesity.