Soccer referees are required to continuously perform perceptual-cognitive and decisional tasks under physical exertion throughout the 90-minute game. They impartially shift from managing to applying the rules in a game (Raab et al., 2020) by making split-second decisions under considerable pressure (Samuel et al., 2020). International soccer referees reportedly make about 137 observable decisions during a single soccer game (Helsen and Bultynck, 2004). An observable decision is when a referee, with or without the support of the assistant-referee, signals or whistles an on-field decision, identifying a foul, an offside, or the direction of a throw-in. However, referee decision making extends beyond making calls consistent with the rules of the game.

As interactors, soccer referees’ decisions are made by the complex interplay between applying the rules and managing the flow of the game, that are contextually grounded in a unique and dynamic environment (Raab et al., 2020). These decisions can be influenced by environmental and cognitive factors, including the referees’ level of experience, field positioning, player and spectator reactions, as well as mental state and concentration level (Lane et al., 2006). Furthermore, the physical aerobic nature of the game could potentially impact their cognitive performance (Castagna et al., 2007). Studies suggest that decision accuracy decreases in the last 15 minutes of a game, presumably related to mental demand and/or physical strain (Mallo et al., 2012; Samuel et al., 2019).

The theorized connection between exercise and cognitive performance has been made through changes in arousal level (Tomporowski and Ellis, 1986). During exercise, heightened arousal can be measured by increased heart rate and/or a greater perceived level of exertion (Easterbrook, 1959). Arousal is related to cognitive performance by an inverted-U shaped relationship where cognitive performance is enhanced by heightened arousal to a critical point that when exceeded becomes detrimental to cognitive performance (Yerkes and Dodson, 1908). Easterbrook’s cue utilization theory is invoked to provide a framework that describes supposed phenomena of this inverted-U relationship (Easterbrook, 1959). The theory proposes that a person’s focus narrows with increasing arousal (i.e. exercise intensity) such that only task-relevant cues will be focused on at moderate levels of arousal, which leads to an improvement in cognitive performance. As arousal increases beyond a moderate level, a person’s focus becomes too narrow so that even task-relevant cues are missed and cognitive performance decreases. Unfortunately, several confounding factors (e.g. task used to evaluate cognitive performance, an individual’s experience/proficiency in decision-making sports) can interact that do not support conjoining theories relating exercise intensity and cognitive performance (Brisswalter et al., 2002). For example, reported gaze behaviours and decision-making in elite association football referees demonstrated that these individuals, without the added influence of physical exertion, may possess enhanced abilities to differentiate relevant from irrelevant information compared to their sub-elite counterparts (Spitz et al., 2016).

A recent scoping review reported contradictory and inconclusive findings regarding the relationship between physical load and referees’ decision-making from different team sports such as Australian football, rugby and soccer (Bloß et al., 2020). Nine of the eleven reviewed studies were conducted in the field and only four studies used physiological data (heart rate in three studies and blood lactate in one study) to quantify physical load (Emmonds et al., 2015; Gomez-Carmona and Pino-Ortega, 2016; Larkin et al., 2014; Mascarenhas et al., 2009). While field studies may provide greater external validity, they cannot control the level of exertion for referees to specifically address hypotheses related to the effects of different levels of exertion on cognitive performance. Bloß and colleagues (2020) suggested that future work should focus on lab-based studies that could standardize physical workload across participants to increase internal validity. Few studies to date have been conducted in lab environments; however, those studies that have controlled the physical exertion point toward no significant relationship with decision-making performance in referees (Elsworthy et al., 2016; Paradis et al., 2016; Samuel et al., 2019). Aside from Elsworthy and colleagues (2016), who used an Eriksen flanker task to assess cognitive performance in Australian football and rugby referees, decision-making in all studies was assessed by either expert video review or video-based decision tests.

Video-based decision-making is commonly used to train and assess referees, mostly limited to video clips with no contextual information, and assessed without physical demand (Samuel et al., 2019). However, decision accuracy varies but considered low (Mascarenhas et al., 2009). Similarly, our (unpublished) findings from pilot work demonstrated poor inter-rater reliability using video vignettes from soccer matches to assess referee decision-making. This informed our decision to use a standardized approach for evaluating cognitive performance in referees. Use of video-review or video-based decision tests, where referee decisions are evaluated against a gold standard of decisions by other referees, to evaluate cognitive performance is another limitation that could be addressed by lab-based studies.

Therefore, the purpose of our study was to elucidate the effect of exercise exertion (Norton et al., 2010) (rest, moderate and maximal exertion) on cognitive performance as measured by the modified Stroop Color Word test (Stroop test) in sub-elite level soccer referees. The Stroop test measures many cognitive properties such as executive functioning and inhibition, selective attention and speed of response (Etnier and Chang, 2009; Pachana et al., 2004), which are related to fundamental aspects of refereeing (MacMahon and Mildenhall, 2012). Consistent with the inverted-U relationship, we hypothesized that cognitive performance would improve with moderate exertion and would deteriorate with maximal exertion.

Participants were recruited from a pool of 50 match-ready referees attending an annual refresher course and who met the Ontario Soccer Association (OSA) Grading Protocol of Grade 8 (OSA match official grade) or higher (Ontario Soccer Assoc, 2019). Grade 8 referees have experience in senior competitive 11v11 games and are eligible to officiate regional or provincial competitions. For the purpose of our study, referees with Grade 8 or higher were classified as sub-elite (Kittel et al., 2019). Participants expressing interest to participate were contacted by email, and provided an outline of the study, consent form and contact information. Those responding affirmatively to the email were then contacted by phone, provided more detailed study information, and if agreeable, scheduled for consent and assessment. The inclusion criteria included: 1) older than 18 years of age; 2) no reported injuries or illnesses preventing participation in intense physical exercise; 3) having successfully completed the Physical Activity Readiness Questionnaire (PAR-Q) (Warburton et al., 2011); and 4) agreed to attend two assessment sessions on separate days. Any participant responding affirmatively on the PAR-Q was excluded from the study. Participants were asked to maintain their normal routines for nutrition and caffeine before each testing session. Testing was conducted during the off-season. The study procedures were approved by the Research Ethics Boards of the Canadian Memorial Chiropractic College (REB# 1508B01) and the University of Toronto (Ref # 32268).

We used a repeated measures within-subject experimental design. All participants underwent two testing sessions that were conducted within 5 to 10 days of each other (Figure 1). Participants were asked not to engage in intense exercise during the 48 hours prior to either of their testing sessions.

Demographic information was collected from each participant at the start of their first experimental session. Next, participants completed a set of five familiarization trials for the assessment of cognitive performance followed by an incremental treadmill-based exertion protocol to determine maximal oxygen uptake (VO_2max Test) as previously reported by our lab (White and Wells, 2015).

For the second session, participants completed a set of three familiarization trials for cognitive performance, followed by an exertion protocol based on the 2015 FIFA fitness test (Canadian Soccer Association, 2014). Cognitive performance was assessed before starting the exertion protocol, at moderate intensity and at high intensity. The moderate and high intensity exercise levels were based on results from each individual’s VO_2max Test results.

Physiological measures and ratings of perceived exertion were obtained from each participant during both testing sessions. Both exertion protocols were completed on the same treadmill (Woodway Desmos, Woodway, Waukesha, WI, USA). Participants were also monitored during both testing sessions for any physical signs or symptoms of exercise complications based on the American College of Sports Medicine guidelines for exercise testing and prescription (Riebe et al., 2015).

Maximal oxygen uptake (VO_2max Test) was assessed during the first experimental session using a standardized incremental exercise protocol (White and Wells, 2015). Workload for the participants was increased incrementally by increasing grade on the treadmill until exhaustion. Specifically, a low-speed warm-up at 0% incline at a self-selected pace (minimum of 6 mph/9.656 km·h^-1) for the first 5-minutes was followed by a multistage run to exhaustion. Treadmill grade and speed for Stage 1 were the same as the warm-up, which lasted for 2-minutes. Stage 2 began with a 2% increase in the grade. The grade was increased by 2% every 2-minutes throughout Stage 2 until the participant completed 2-minutes of running at an 8% grade. Stage 3 involved 2% increases to the grade every minute until the participant reached volitional exhaustion.

The exertion protocol for the second experimental session was based upon the 2015 FIFA fitness test. The protocol began with a self-selected low-speed warm-up at 0% incline on the treadmill for the first 5-minutes. Immediately following the warm-up, participants began running at the same self-selected speed that was used at the start of their VO_2max Test. The participants alternated between 35-seconds of running at 0% of incline, and 30-seconds of exercise at the incline reached when the participant indicated volitional fatigue at the end of their VO_2max Test. Running speed during the exertion protocol was increased by 0.5 mph/0.805 km·h^-1 every 5-minutes until the participant reached volitional exhaustion.

Cognitive performance was assessed using the Stroop Test. This test is frequently used in neuropsychological and psychometric studies to assess cognitive performance and has acceptable psychometric properties (Siegrist, 1995; Siegrist, 1997). The short version of the interference Stroop Test was used to mitigate against the possibility that participants would begin recovering from the exertion protocol as they were doing the Stroop Test. The short version consisted of 42 color-words (stimuli) that were distributed in a matrix of 6 columns and 7 rows. It has been suggested that the shortened Stroop Test provides similar results as the longer version and is easier to administer (Klein et al., 1997). The stimuli consisted of the name of a color that was printed in a different colored font (Figure 2).

Participants completed all trials of the Stroop Test while standing on the treadmill. Lighting was dimmed during each iteration of the test within the stimuli matrix projected on the wall that was 3 meters in front of the participant. Participants were instructed to accurately name the font color as quickly as possible (Jensen and Rohwer Jr, 1966). Cognitive performance was operationalized by the time to complete the Stroop Test, which was measured with a stopwatch. Errors made by participants were not monitored; however, consistent with previous work, participants adopted a practice of automatically self-correcting errors (Jensen and Rohwer Jr, 1966; Klein et al., 1997).

Participant rating of perceived exertion (RPE) was obtained throughout the exertion protocol on both days of testing by using the Borg scale (Borg, 1982). A RPE on a scale from 6-20 was obtained before each increase of the treadmill grade in the VO_2max assessment during the first testing session and it was obtained at the end of every second interval (after the incline running) throughout the exertion protocol during the second experimental session. Moderate exertion was defined as the grade and speed where the participant first reported a RPE of 13. Maximal exertion was defined as the point where the exertion protocol was ended and corresponded to an RPE level if 18-20. The participants reported RPE values of 18-20 at volitional exhaustion, which is consistent with previously reported exercise session RPE values in professional rugby players (Weaving et al., 2017).

Oxygen consumption (VO₂ (L/min)) was measured during the first session using a Metamax portable breath-by-breath automated metabolic system (Metamax 3B, Cortex Biophysik, Germany). Prior to exercise testing, measured ventilation was calibrated using a standard 3L syringe and respiratory gases were calibrated using a two-point calibration of ambient room air and a known gas mixture consisting of 15% O₂ and 5% CO₂. VO_2max was taken as the mean VO₂ achieved during the last minute of exercise prior to volitional exhaustion.

The participants’ heart rate (HR) was continuously monitored during the exertion protocol on both days using a telemetered heart rate monitor (Polar T34, Polar Electro Inc., New York, USA). Maximum HR achieved during each exertion protocol was used for subsequent analysis.

Blood samples were also obtained before and after the exertion protocol on both experimental days. To obtain a blood sample, the participants’ fingers were cleaned with an alcohol swab and the fingertip was lanced. The first blood droplet was wiped clear, allowing for an uncontaminated sample from the second blood droplet. Blood samples were acquired immediately before, and 2-minutes after completing each of the exercise tests. All blood samples were immediately analysed for lactate concentration ([La-]_b) using the i-STAT Portable Clinical Analyzer (Abbott, NJ, USA)(Verheijen et al., 1999).

Time to complete the Stroop test before, during and after the exertion protocol during the second session was analysed by a single-factor within-subjects repeated measures analysis of variance (ANOVA). Data collected from the Stroop test familiarization trials were not used for statistical comparisons; however, descriptive measures (i.e. means and standard deviations) were calculated for the time to complete the Stroop test in each of the familiarization trials. [La-]_b was evaluated by a two-factor (DAY and TIME) repeated measures ANOVA. A two-tailed paired samples T-test with a significance level of 0.05 was used to compare maximum HR achieved during the exertion protocols on Day 1 and Day 2. Post hoc analyses of statistically significant main and/or interaction effects from the ANOVA were performed by paired comparisons of means. Bonferroni adjustments were applied to the significance level for all post hoc statistical analyses. After adjustment, the statistical significance levels for the post hoc tests were set at 0.013 for [La-]_b, and 0.017 for completion time on the interference Stroop test. Cohen’s d was calculated to determine effect sizes (ES), which were interpreted as small (0.2 ≤ d < 0.5), medium (0.5 ≤ d < 0.8) or large (d ≥ 0.8) according to Cohen’s recommendations (Cohen, 2013). All statistical analyses were performed using SPSS software (SPSS Inc., Chicago, IL, USA).

Twelve healthy participants (8 males and 4 females) volunteered. Group demographics of the participants are summarized in Table 1. No adverse effects of either exertion protocol were noted.

Performance on the Stroop test significantly improved relative to rest at both moderate and maximal exertion on the exertion protocol conducted during the second experimental session (Figure 3). Post hoc comparisons revealed a significant improvement between baseline (pre-exercise at rest) performance and at the point of moderate exertion (p = 0.005, d = 0.77) and between baseline and maximal exertion (p = 0.001, d = 1.09). A non-significant decrease, considering the aforementioned Bonferroni correction, in Stroop test time was found between submaximal exertion and maximal exertion (p = 0.035, d = 0.48).The maximum HR reached during the VO_2max test was higher than during the exertion protocol (p = 0.025, d= 0.51) (Figure 4). [La-]_b also increased after exercise compared to rest on both days (p< 0.001, d ≥ 3.16) (Figure 5); however, post-exercise [La-]_b was 28% lower after the exertion protocol than after the VO_2max test (p < 0.001, d= 1.40). There was no significant statistical difference found in mean [La-]_b before the VO_2max test and the exertion protocol (p = 0.803).

The current study assessed the immediate effect of cardiovascular exercise intensity (rest, moderate and high intensity) on cognitive performance in sub-elite level soccer referees. Our findings demonstrated that cognitive performance improved with moderate intensity of physical exertion. Furthermore, this improvement in cognitive performance was retained immediately following high intensity physical exertion. These findings provide mixed support for our hypotheses that were based on the conjoined theories (inverted-U and Easterbrook’s cue utilization) linking arousal to cognitive performance.

There are many possible reasons for why we did not observe a decrease in cognitive performance at high exertion. Two of these reasons are described below. Despite maximum self-reported intensity being the same between exercise protocols on each day of data collection (score of 18 on the Borg scale), physiological measures of intensity (maximum HR and blood lactate) were lower for the exertion protocol on Day 2 of data collection than the VO_2max test that was completed on Day 1. The exertion protocol on Day 2 of data collection was meant to lead to high intensity exertion, a physiological state that can happen to an elite referee after consecutive sprints or high intensity runs in a game situation (Castagna et al., 2007). In comparison, the VO_2max test was used to assess maximum HR and cardiovascular fitness (White and Wells, 2015). Our findings suggest that the exertion protocol on Day 2 of data collection may not have enabled participants to attain maximal exertion, rather only high intensity exertion comparable to the intent of the 2015 FIFA fitness test. Brisswalter and colleagues noted that cognitive performance on decisional tasks improved in a range of exercise intensities between approximately 40% to 80% VO_2max (Brisswalter et al., 2002). Thus, it is possible that the exertion level of referees at the end of the exertion protocol on Day 2 in our study was still within the region where enhanced cognitive performance is observed. Another potential explanation for our finding of enhanced cognitive performance at high exertion is that referees belong to a specialized population that are accustomed to making decisions while performing physical activity. Perhaps awareness of their physical fatigue triggers a self-imposed attentional strategy altering their decision-making focus. This is supported by previous research demonstrating that cognitive performance may be differentially altered by physical exertion in athletes that participate in or are more proficient at decision-making sports (Delignières et al., 1994; McMorris and Graydon, 1996).

As previously mentioned, a recent scoping review reported that there is currently mixed evidence regarding the relationship between physical workload and decision-making in soccer referees (Bloß et al., 2020). Two of the future directions offered in the scoping review were addressed by the current study, which were directed toward improving internal validity. Specifically, our study attempted to standardize the physical exertion and workload across the referees along with obtaining physiological measurements of internal workload. On average, participants achieved a peak heart rate of 187 bpm, which is similar to previously reported values from match referees (Castillo et al., 2016; Krustrup et al., 2009); however, blood lactate measures at the end of the exertion protocol on Day 2 of data collection were larger than previously reported post-match blood lactate measures in match referees (Castillo et al., 2016; Krustrup et al., 2009). Also different from the previous studies, we chose to measure cognitive performance by administering the Stroop test instead of relying on expert video-review or video-based decision tests. This choice was made on the basis of our (unpublished) pilot testing that revealed poor inter-rater reliability of decisions from soccer vignettes. The Stroop test was chosen as the decision-making task to evaluate cognitive performance in the current investigation due to its favourable psychometric properties, ease of administration and extensive use in neuropsychological research for evaluating executive function (Etnier and Chang, 2009; Siegrist, 1997).

Our study had a small sample size. Sub-elite soccer referees are a very unique study population. In 2015, there were 50 sub-elite soccer referees (with a provincial grade or higher) registered in Ontario. Out of those 50 referees, 33 resided within the geographical catchment area, of whom 36% took part of in our study. However, based upon the sample of 12 subjects, our observed power for the difference between baseline and submaximal exertion was 0.43, and the observed power for the difference between baseline and max exertion was 0.80.

A second limitation was related to our methodological decision to not obtain data from a group of participants who were not referees. Pragmatically, participants of a control group would have needed to have been matched to the referees on multiple factors such as sex, age and fitness level. This would have substantially increased the effort and cost of data collection. Nonetheless, it is uncertain if our conclusions are specific to the population of referees or if these findings are more generally representative of the effects of cardiovascular exertion. Future studies could replicate our protocol with an appropriate sample of non-referees to determine if the findings are specific to referees or representative of a more general inference relating cardiovascular exertion to cognitive performance.

Participant familiarity (or lack thereof) with the interference Stroop test was another possible limitation. It has been shown that during the first trials of a Stroop test, subjects get better and faster (practice effect), after which they reach a certain performance plateau (Jensen and Rohwer Jr, 1966; Klein et al., 1997). A potential learning effect of performing the Stroop test was also observed in a crossover study comparing changes in cognitive performance after a sedentary task and after an exercise task (Sibley et al., 2006). We used a familiarization session of 8 trials, across two testing sessions, before testing to minimize the effect of learning; however, the number of practice trials needed to reach that performance plateau is unclear (Jensen and Rohwer Jr, 1966; MacLeod, 1991). Although not statistically evaluated for significance, our data from the practice trials appeared to demonstrate a learning effect for the Stroop test. Moreover, this learning effect carried over into Day 2 of experimentation and even demonstrated a continued improvement from Day 1 to Day 2. However, no further learning effect was observed over the three practice trials and baseline in Stroop time, suggesting a performance plateau was reached on Day 2.

We elected to choose the exercise modality that would most closely reflect the exercise demands of a referee on the actual soccer field. We therefore chose running as the exercise for this research. It is likely that other forms of exercise would present different cognitive demands on the participants, for example balance would not be an issue while cycling on a stationary ergometer. However, we do think that running on a treadmill is the closest approximation of the conditions that a referee would experience in the field in a game situation. Note we did not specifically ask the referees if they were used to running on a treadmill; however, none of the participants exhibited difficulty with the running task during the protocols.

It is possible that external factors, such as the crowd and players’ reactions, importance of the game, as well as the referees’ mental fatigue and state, may have a more important or cumulative impact on referees’ cognitive performance than physical exertion at the end of games (Lane et al., 2006). Our study took place in a controlled environment and only assessed referees’ cognitive performance through one parameter, which was time to completion of the Stroop test. In our experimental scenario, attention narrowing was beneficial to referees. However, in real-game situations, elite soccer referees must be aware of their entire on-field surroundings and do not focus on only one component of the game (Catteeuw et al., 2009). Thus, attention narrowing could lead to a decrease in their overall on-field performance.

Findings from the current study add to the growing body of literature on the relationship between physical workload and decision-making in referees. The results of our study suggest that acute moderate and high intensity exercise may improve cognitive performance in sub-elite soccer referees as assessed with the interference Stroop test. These findings are supported by previous research using similar outcome measures and interventions to assess the impact of exercise on cognitive performance (Brisswalter et al., 2002). Future studies should incorporate a more realistic game environment to determine the impact of non-exertional factors on cognitive performance.