Resistance training, commonly referred to as strength training or weight training, is considered an essential intervention for improving sports performance in most sports. There are many resistance training modalities, including free weights, machine weights, isokinetic devices, elastic bands, resisted running, and plyometrics. Resistance training has been repeatedly shown to be highly effective at increasing strength, speed, endurance, and sports performance in a variety of athletic populations (Seitz et al., 2014; Balsalobre-Fernández et al., 2016; Davies et al., 2017; Kwok et al., 2021). Therefore, it is not surprising that coaches are increasingly interested in resistance training methods whose effectiveness has been demonstrated by rigorous evaluation in high-quality studies. Evidence-based practice in sport sciences is a paradigm that promotes the integration of the practical experience of coaches and practitioners, athletes’ values, and high-quality studies into the decision-making process for athlete care. Practitioners utilizing resistance training as a modality should apply the best scientific evidence to their training programs that are well-designed and well-reported (Amonette et al., 2016).

One approach to meeting these quality challenges is to take the appropriate steps in design, execution and reporting of research to increase the internal validity of investigations. Specifically, internal validity in experimental and quasi-experimental research designs indicates the degree to which changes in a dependent variable can be attributed to changes in an independent variable (Taylor and Asmundson, 2008).

Several basic methodological procedures are required to ensure high internal validity. Random assignment of participants to experimental and control groups (randomization) is a gold standard for experimental research, as it decreases the risk of selection bias by equally distributing the study participants with particular characteristics among all groups. Simple randomization, block randomization, and stratified randomization are the most common randomization techniques (Lin et al., 2015). It is interesting to note that estimates of intervention effects tend to be smaller in publications where random allocation is clearly reported, compared to works when non-random or quasi-random methods are used (Savović et al., 2012). A randomized experiment is an essential tool for testing the effectiveness of the intervention.

The second important feature for randomization procedures, therefore to internal validity is allocation concealment. Concealed allocation is adopted when the researcher is unaware of the sequence for group allocation. When an allocation is not concealed, the researcher (recruiter) may consciously or unconsciously influence allocation to a particular group, increasing the potential for selection bias which may taint the data. Schulz and Grimes (2002) determined that experimental research with inadequate or unclear allocation concealment tended to overestimate treatment effects, up to 40%, compared with those where concealment was used adequately. In addition, Schulz et al. (1995) reported that studies where allocation concealment was poor showed greater heterogeneity in results, due to more extensive fluctuation between above and below the estimates, relative to studies where allocation was both clear and adequate.

Examining similarity of the experimental and control groups at baseline is another important criterion for internal validity. This procedure allows the researcher to compare the groups on variables of interest following randomization to determine whether the groups are equivalent. Because it is still possible that some differences will occur, the knowledge about the magnitude of potential differences between groups at baseline is important for reliable interpretation of data from an investigation. Typically, demographic (e.g., age, sex) and anthropometric (e.g., height, body mass) variables, as well as key outcome measures for the study, are included in this analysis. Note that using the statistical control for baseline data across the randomized groups is often discouraged (Schulz et al., 2010). For example, according to the Consolidated Standards of Reporting Trials (CONSORT) guidelines for individually randomized trials, baseline statistical testing should not be applied, because based on the assumption of randomization, it is known that any baseline differences are caused by chance (Schulz et al., 2010). On the other hand, some researchers claim that baseline testing allows them to check that the randomization process has not been subverted (Berger, 2005).

Blinding is a procedure that reduces the risk of detection and performance bias by preventing subjects and researchers who are involved in the study from knowing to which group a participant was allocated. However, in strength and conditioning research, where differing exercises or training programs of often evaluated, both participants and supervisors (training instructors) are actively engaged in an intervention. In such cases, blinding is not completely possible. This limitation appears in these types of studies because participants and supervisors may consciously or unconsciously influence study results. For example, researchers may more efficiently motivate experimental group participants to confirm the hypothesis they set out to test. This is concerning for correct application of evidence-based practice in strength and conditioning, as there is evidence that lack of blinding leads to overestimated intervention effects (Jüni et al., 2001). While it is impossible to eliminate the risk of performance bias in studies of this type, reducing the risk of detection bias is possible by blinding the researchers who measure outcomes along with those who collect and analyze data.

Another methodological common challenge is attrition of participants during study. This threat to internal validity refers to the differential and systematic loss of participants from experimental and control groups. When attrition occurs, the characteristics of randomized groups may change from the initial allocation, and these changes may affect the study outcomes in an uncontrolled manner (Beutler and Martin, 1999; Dumville et al., 2006). One study that analyzed the effects of treatment in 235 randomized control trials (RCT) published in leading medical journals found that different assumptions about outcomes of participants who withdraw from the study could change interpretation of results of up to 58% of RCTs (Akl et al., 2012). Attrition may occur for numerous reasons, including diseases, participant loss of interest or poor tolerance for an intervention. In strength and conditioning research, there is also the potential for participants who are athletes to be injured in practice or competition outside of the intervention being used in the study. Even if the intervention is highly tolerable, some participants may not adhere to the allocated intervention due to the intervention is perceived as ineffective or they are dissatisfied with their allocated intervention. An athlete, for example, might fear loss of a competitive advantage if they are placed in a group which receives an intervention they believe may be less effective, causing them to withdraw from the study.

To mitigate attrition bias, different statistical techniques have been applied. Intention-to-treat (ITT) analysis is considered the gold standard method for dealing with attrition in RCT studies. According to the basic assumption of ITT, all randomized participants are included in the analysis in accordance with group assignment, regardless of their adherence, intervention duration or change of intervention regimen. As a result, data is analyzed irrespective of the planned study protocol. If the attrition rate is relatively high (e.g. >20 %), ITT analysis tends to underestimate the intervention effects in participants who complete the study (Armijo-Olivo et al., 2009).

Study quality assessment is often performed with the use of an assessment tool. No formal recommendation regarding such tool for strength and conditioning research exists, leading researchers to sometimes adapt tools from other, related, disciplines (Smart et al., 2015). One of the most common used assessment tool for rating the internal validity of primary articles included in systematic reviews is the Physiotherapy Evidence Database (PEDro) scale (Maher et al., 2003). The PEDro scale is based on the Delphi list developed by Verhagen et al. (1998) and, as its name suggests, was initially designed for physiotherapy-based studies. Although some criteria of the PEDro scale are redundant for strength and conditioning studies, and others relevant for training intervention-effectiveness (e.g., adverse events, training frequency, volume and intensity of training) are not addressed, the PEDro scale does allow for the assessment of essential features of internal validity for strength and conditioning research.

The authors of this paper take a view that the PEDro scale items may also provide distinct advantages for primary research like RCT studies (outside the context of a systematic reviews) (Albanese et al., 2020). For example, by using them as guidelines during the planning stage of study design, researchers may minimize the influence of confounding variables, improve methodology, and receive more reliable information about experimental interventions. Numerous studies (Harries et al., 2015; Blagrove et al., 2018; Thiele et al., 2020; Trowell et al., 2020) suggest that internal validity of strength and conditioning research could be higher. However, because the results are not conclusive, we decided to review methodological quality of strength conditioning research which examined the effects of resistance training on strength, speed, and endurance in athletes. Our goal was not to provide a review of the total quality assessment of studies as previous systematic reviews have done, but rather focus on internal validity criteria to provide more attention to methodological procedures which should be used during planning and reporting of experimental research in the field of strength and conditioning. In addition, we believe that a greater depth of understanding of research quality by practitioners may support the process of using the best evidence in practice.

The PRISMA guidelines for reporting a systematic review were adopted (Moher et al., 2009). As this study did not involve human subjects, institutional review board approval was not required. Four relevant electronic databases (SPORTDiscus, Scopus, Medline, and Web of Science) were comprehensively searched for studies examining the effects of resistance training on strength, speed, and endurance in athletes. The identified terms with Boolean operators with different expressions are presented in Table 1. All publications listed prior to April 15, 2021 were considered for inclusion without language restrictions.

Two independent investigators (HM, MS) screened titles, abstracts, and full-text articles against the PICO criteria (Table 2). Following extraction, duplicate articles were removed automatically using EndNote X9.3.3 (Clarivate Analytics). Any remaining duplicates were deleted manually. In any disagreement regarding inclusion/exclusion, ambiguous issues were discussed, and a consensus was reached before proceeding.

The internal validity of each study included in this review was assessed using the PEDro quality scale. This appraisal tool was chosen because it demonstrates high reliability and validity for randomized control trials (Maher et al., 2003). In addition, the PEDro scale makes it possible to assess the clarity of statistical methods and report external validity criteria. A given PEDro item was scored as a “yes” if the criterion was met and the item scored as a “no” if the criterion was not met. Points were awarded only when a given criterion was satisfied according to the PEDro guidelines (please see https://pedro.org.au/english/resources/pedro-scale/; Maher et al., 2003). Note that the PEDro scale was adapted to the methodological and reporting requirements of strength and conditioning filed in this study. Criterion 1 “eligibility criteria were specified” was satisfied if the study reported the source of subjects and a list of criteria used to determine who was eligible to participate in the study. Criterion 2 “subjects were randomly allocated to groups” was satisfied if study stated that allocation was random. The precise method of randomization was not needed to specified in order to satisfy this criterion. However, quasi-randomization research did not satisfy this criterion. A point was awarded for criterion 3 “allocation was concealed” when authors of a study stated that group allocation was concealed or, when it was stated that the researcher who determined if a subject was eligible for inclusion in the study was unaware, when this decision was made, to which group the subject would be allocated. Criterion 4 “the groups were similar at baseline regarding the most important prognostic indicators” was considered to have been met if a study reported at least one key outcome (primary measure of the effectiveness of the training intervention) and anthropometric variables such as body mass, body height or one repetition maximum test at baseline. Measure of the severity of the condition being treated was not applicable in this study. Criterion 4 was met if baseline data were presented by group allocation, and when there was no difference between prognostic indicators. Criterion 5 “there was blinding of all subjects” and criterion 6 “blinding of all therapists who administered the therapy” were not applicable in this study. A noted earlier, in strength and conditioning research, both participants and those applying a particular intervention are aware of which intervention is being applied. Criterion 7 “there was blinding of all assessors who measured at least one key outcome” was met when it is stated that the assessor of the primary outcome was blinded to group allocation. Criterion 8 “measures of at least one key outcome were obtained from more than 85% of the subjects initially allocated to groups” was satisfied only if the study explicitly reported both the number of subjects initially allocated to groups as well as the number of subjects from whom key outcome measures were obtained. In studies in which outcomes are measured at several points in time, a key outcome must have been measured in more than 85% of subjects at one of those points in time. Criterion 9 “all subjects for whom outcome measures were available received the treatment or control condition as allocated or, where this was not the case, data for at least one key outcome was analyzed by intention-to-treat” was met if intention-to-treat analysis was performed or, if the study explicitly demonstrated that all subjects received the training intervention or control condition as allocated. Criterion 10 “the results of between-group statistical comparisons are reported for at least one key outcome” was satisfied when comparison of two or more training interventions, or comparison of training intervention with a control condition was applied and when a factorial analysis of variance or hypothesis testing, describing the probability that the groups differed only by chance or in the form of an estimate (e.g., the mean or median difference) and its confidence interval have been used to analyze the data. Criterion 11 “the study provides both point measures and measures of variability for at least one key outcome” was met when a point measure was a measure of the size of the treatment effect (e.g., described or presented as a difference in group outcomes), or as the outcome in (each of) all groups and measures of variability included standard deviations, standard errors, confidence intervals, interquartile ranges (or other quantile ranges). Two qualified PEDro raters independently evaluated all included studies (HM, MS). All differences of opinion regarding the PEDro rating were discussed with a third independent qualified PEDro rater (MP).

Figure 1 shows the flow chart of the study selection process. The systematic literature search identified 6,516 records, of which 2,270 duplicates were removed. Screening for title and abstract identified 211 possibly relevant studies. After the full-text screening, 133 studies were selected for rating by the PEDro scale.

The characteristics of the included studies are summarized in Table S1 (see Supplemental File Table S1, the references of the included studies are also listed in this Supplemental File). The total sample size was 3,117 subjects (n = 2,532 men, 81%; n = 585 women, 19%). The subjects were athletes between the ages of 18-65 years who engaged in organized sports training and competition. Across all studies, 74% of participants represented team sports (e.g., American football, basketball, handball, rugby, soccer), and 26% of participants represented individual sports (e.g., cross country skiing, cycling, track and field, tennis). Overall, 54 studies reported sports experience (range: 0 to 14 years) with the remaining 79 not explicitly describing subjects’ level of experience.

Training program durations varied from 4 to 40 weeks. The training interventions consisted of different types of resistance training, including traditional heavy resistance training (e.g., weight training, free weight training, squat training, eccentric exercises), resistance power training (e.g., Olympic lifting, explosive strength training, high-velocity resistance training), plyometrics (e.g., horizontal, vertical jumps, aquatic plyometrics), assistance exercise (e.g., core exercise, elastic bands), machine weights (e.g., machine squat jump training, exercise with isokinetic device) and resisted running (e.g., sled towing).

Most studies evaluated running speed with short linear sprints ranging from 5 to 50 m, including repeated sprint ability tests. Strength evaluation was mainly based on the one-repetition maximum (1RM) test of a back squat (full or half squat variations), bench press, and/or deadlift exercises. The included studies also assessed strength performance with 1RM tests for the pull-over, leg-press, power clean, snatch, clean and jerk, hip thrust, seated lat pull-down, chop-test, cable pulley, step-up, military press, lunge, shoulder flexion, and shoulder abduction. Six studies evaluated resistance training interventions on the improvement in endurance performance with running-based time trials. These outcomes were evaluated at distances ranging from 800 to 5000 m.

The evaluation of internal validity items across the 133 studies is illustrated in Table 3 (for complete data, see Supplemental File Table S2). Although 68% (n = 90) of the included studies used random allocation to groups (Criterion 2), only 0.8% (n = 1) reported concealed allocation (Criterion 3). Baseline data (Criterion 4) are presented in 69% (n = 92) of the studies included in this review. Thirty-eight percent (n = 51) of studies showed adequacy of follow-up of tested athletes (Criterion 8). The plan to follow intention-to-treat (Criterion 9) or, stating that all participants received training intervention or control conditions as allocated, were reported in 1.5% (n = 2) of the studies we reviewed. Blinding of assessors (Criterion 7) was satisfied in 1.5% (n = 2) of the studies. None of the studies met the blinding of subjects and training supervisors’ criteria.

The eligibility criteria (Criterion 1) were specified in 22% (n = 28) of studies. Reporting of results of between-group statistical comparisons (Criterion 10) and point measures and measures of variability (Criterion 11) were included in 85% (n = 113) and 96% (n = 128) of studies, respectively.

This systematic review aimed to provide a comprehensive overview of the procedures that were used to ensure internal validity in resistance training research in athletes. This review revealed strengths and limitations of study designs and reporting procedures in strength and conditioning research. The main findings of the current study suggest that internal validity, as assessed by particular PEDro scale items, varied from moderate to very low. Accordingly, there are methodological safeguards which should be widely adopted in experimental studies in strength and conditioning to improve internal validity.

Randomization is fundamental to the design and conduct of experimental research, the current analysis of resistance training research showed there is still room for improvement. Thirty-two percent of analyzed studies used less rigorous, often quasi-experimental design, in which there is only one group or in which randomization to more than one group is lacking (Balsalobre-Fernandez et al., 2013; Grazioli et al., 2020; Bachero-Mena et al., 2021). Quasi-experimental design does not control for unwanted systematic differences (selection bias) between groups, therefore it is a potential a threat to internal validity. This limitation occurs quite often in strength and conditioning research due to athletes’ or coaches’ training preferences. For example, coaches may be concerned about a reduction in preparedness for their athletes who are assigned to an intervention that they perceive to have less efficacy. There is also an ethical problem, if participant in the control group miss out taking part in experimental intervention which usually are considered as more beneficial than the intervention (or the lack of intervention) assigned to control group. To deal with these constraints, alternatively, researchers may consider the use of crossover or within-subject designs, where each participants of the study receive a series of interventions in random order and the outcomes are uniquely associated with each intervention.

Sixty-eight percent of the included studies were considered to have a random allocation, but only a few of them provide information about the techniques of randomization utilized (Anderson et al., 2008; Impellizzeri et al., 2008; Ali et al., 2019). From a methodological assessment of randomization procedure perspective, providing information about which technique was utilized in the study is important. For example, it is known that simple randomization is suitable for large samples of participants (n > 100), but not appropriate when the size of sample is small, which is very common for strength and conditioning research – on average, 23 participants took part in each study in the present review. In such cases, utilizing of stratified randomization is recommended. This method allows for control and balancing the influence of covariates. The prerequisite for stratified randomization is that researchers should be able to identify each of the covariates which potentially influence key outcomes prior to group allocation. Common covariates for evaluation of effectiveness of resistance training interventions are age, sex, body mass, and training status. Considering above arguments, we believe that providing details whether, and how randomization was performed is important for internal validity of studies in strength and conditioning.

Despite data which suggests that lack of allocation concealment may improperly lead to a larger effect for a particular intervention (Odgaard-Jensen et al., 2011), only one of the included studies reported concealed allocation (Grazioli et al., 2020). This is consistent with data presented on the Physiotherapy Evidence Database (PEDro) (https://pedro.org.au/wp-content/uploads/11Jan2021.pdf). However, given the ease of implementation and reporting of concealed allocation, which does not seem to present the challenges in strength and conditioning research which have been noted in other criterion, the lack of intention to this is concerning. Concealed allocation is a safeguard against researchers consciously or subconsciously introducing systematic differences in groups (Elkins, 2013). For example, when the effectiveness of a particular training method is being tested, athletes who are anticipated to have low adherence, or perhaps those athletes who are considered to be of a lower level by coaches or investigators, may be delayed until the probability of allocation to a control group is greater. Therefore, performing and reporting of concealed allocation in strength and conditioning research are required.

Randomized allocation to intervention and control groups does not guarantee that the groups are similar at baseline. Therefore, it is recommended to compare a few variables before the intervention to investigate the whether the groups are comparable. According to the PEDro scale, an article should provide data for each group for at least two variables, one measure of severity, and at least one (different) key outcome, which provides a measure of effectiveness or lack of effectiveness of the intervention. This criterion is only satisfied if baseline values are presented. Since strength and conditioning research typically includes healthy participants, the authors of this review replaced the "measure of severity of condition treated" with measures more related to the specifics of strength and conditioning studies, namely body mass and height. There is large of body research that indicate that body characteristics may influence resistance training outcomes (Twist et al, 2021). Almost 70% of the included studies reported these data, mainly in a table or figure. However, reporting of information on other baseline variables which may potentially influence the outcomes of resistance training intervention, such as results of one repetition maximum (1RM) testing, was much less common (Grgic et. al, 2020). This is concerning, as the training status of athletes would seem to clearly influence the potential for adaptation to a given intervention.

Interestingly, although using statistical significance tests for baseline data is not recommended (Schulz et al. 2010), statistical testing for judging the baseline comparability was common in the included studies. According to recommendations published in a top medical journal, The Lancet, it is sufficient for baseline characteristics to be reported and compared using descriptive statistics with a mean and a measure of variability (Schulz and Grimes, 2002). In addition, Schulz and Grimes (2002) proposed that continuous variables (e.g., age, body mass) may be reported as a mean and standard deviation. However, when data distribute asymmetrically, a median and percentile range (interquartile range) are more appropriate (Schulz and Grimes, 2002).

Blinding of participants and training instructors who involved in resistance training research is not realistic. However, those who are assessing and interpreting outcomes, including: data collectors, judicial assessors of outcomes and data analysts should be blinded to increase internal validity. Only 1.5% (n=2) of the studies included reported this procedure. These results are consistent with those of other reviews of resistance training interventions, which suggest that this criterion of internal validity is often not met (Thiele et al., 2020; Trowell et al., 2020). When assessors are blinded, they may be less likely to have conscious or unconscious biases affect outcome assessments due to their expectations or beliefs about the effectiveness of the intervention. There are several strategies aimed at blinding assessors in strength and conditioning training (e.g. preventing assessors from accessing data which has the potential to compromise blinding).

Withdrawal and dropout of participants provide various challenges to investigators due to changing the baseline characteristics between the randomized groups. High rate of attrition and uncontrolled attrition have high potential to harm internal validity of outcomes. According to the criterion of the PEDro scale, a high risk of attrition appears when the publication does not explicitly state both the number of subjects who were initially allocated to a group and when key outcomes have not been measured in more than 85% of subjects. It should be noted that authors of the included studies in strength and conditioning often did not explicitly report the number of participants who completed the study. Adequacy of follow-up was fulfilled only by 40% of the studies. As this is an issue of reporting rather than design, this is simple to improve in future investigations. Attrition from research training programs in athletes may be due to several potential reasons, such as injury, concern for injury, loss of motivation to participate in research, or scheduling conflicts with their training program. To prevent high attrition rate and increase compliance with the protocol for the assigned groups, the research should be attractive and beneficial for participants and coaches of these athletes. It is good practice to include coach into research team.

The other method of reducing attrition bias is to implement an intention-to-treat analysis (Moseley et al., 2011). Employing the intention-to-treat procedures is not difficult. Firstly, the subjects whose participation in training intervention has been interrupted should be encouraged to participate in the remaining outcome measurements, if possible. Secondly, statistical analysis should reflect the allocation design and consider all obtained data (Elkins and Moseley 2015). Note that several methods have been identified to fill in the missing data (Nakai et al., 2014; Smart et al., 2015). In our review, intention-to-treat was undertaken only in 1.5% (n=2) of all included studies (Nonnato et al., 2020; Richards and Dawson, 2009).

The PEDro scale items also assess if statistical information reported in a study are interpretable. To fulfil this criterion, a study needs to report between-group comparisons. The second criterion is satisfied when a study reports both a measure of the size of the intervention effect and a measure of variability for at least one key outcome. As noted earlier, our findings showed that most of the reviewed studies demonstrated sufficient information to make them interpretable, 85% for criterion 10 and 96% for criterion 11.

The third, outside internal validity, criterion relates to external validity. This criterion is considered to have been met if a publication describes the source of the subject pool and when a list of inclusion or exclusion criteria of participants in the study is reported. Although reporting the source of participants of the study and specified eligibility criteria is an important attribute of generalizability of the study findings and is easy to apply in resistance training research, only one in five articles in the current review included this information. These results are consistent with those of other systematic reviews which investigated the effects of resistance training on the performance of athletes (Thiele et al., 2020; Trowell et al., 2020). For example, Trowell et al. (2020) found that 75% of included studies did not list eligibility criteria for participants. To address this, and the threats mentioned above to internal validity, researchers may wish to consider the use of CONSORT (Consolidated Standards of Reporting Trials) guidelines (Schulz et al., 2010).

This systematic review provides an overview of the literature regarding internal validity procedures in resistance training studies. Nonetheless, some limitations should be considered. An investigation of internal validity was based only on criteria included in the PEDro scale. Other variables may need to be considered when validity in resistance training research is investigated. From the present study, it is impossible to determine if internal validity limitations in the included studies were caused by shortcomings in the study design or reporting procedures. The sample used in the present review was limited to athletes; therefore, these findings may not be generalizable to studies involving non-athletes.

Although the PEDro scale is used to evaluate the quality of randomized controlled trials in systematic reviews, our findings showed that particular items of the PEDro scale may be used as a guideline in study design and conducting experimental research in strength and conditioning. The largest threats to internal validity in the studies we analyzed were associated with concealed allocation, intention-to-treat, and blinding of assessors of the main outcomes. Further improvement in the quality of studies should also involve random allocation to groups, ensuring that groups are similar at baseline, measures of key outcomes from the highest number of the subjects according to their initial allocation to groups, as well as one external validity variable (specification of eligibility criteria).

Because the current review showed that internal validity items like concealed allocation, intention to treat, and blinding of assessors of the main outcomes are often not reported in resistance training studies, the internal validity of future studies should be improved. Well-designed, well-conducted, and well-reported experimental research studies are essential to confirm whether training interventions improve outcomes as poorly designed and reported studies can mislead decision making in professional practice.