Running is a popular sport and leisure-time physical activity across the globe and among different age groups (Hulteen et al., 2017). Adhering to a well-designed and evidence-informed training plan is crucial to increase the likelihood of optimal biomechanical, psychological and physiological adaptations aimed to improve running performance (Düking et al., 2020b), and to decrease the likelihood of (overuse) injuries and adverse health effects. Yet, the majority of runners especially at a novice level, either do not have sophisticated knowledge on how to design (evidence-informed) training plans or lack access to coaches providing individually tailored training plans (Vos et al., 2016).

To overcome these limitations, Artificial Intelligence (AI) might be one solution to provide individuals with training plans at large scale. ChatGPT is one AI software (more specifically a large language model) which has gained widespread attention and reached 100 million users within 64 days following its release on November 30, 2022 (Hu, 2023). It is designed to engage in interactive conversations with users, providing human-like responses based on the input it receives. ChatGPT utilizes deep learning techniques to process and generate responses, and has been trained on a large dataset of human conversations. This allows the AI to understand and generate natural language text.

The availability of advanced AI capabilities has spread from a few skilled experts to a wide range of people, leading to the discovery of various unforeseen applications. Currently, there is ongoing research to assess the effectiveness of ChatGPT in healthcare applications (Ayers et al., 2023; Lukac et al., 2023; Tsui et al., 2023). One area of investigation focuses on its ability to deliver compassionate and reliable responses to patients seeking medical information. For instance, a recent study evaluated ChatGPT´s capacity to offer empathetic and accurate answers to healthcare-related inquiries (Ayers et al., 2023). This investigation involved comparing ChatGPT´s answers with responses provided by physicians on a public social media forum and showed that ChatGPT generated quality and empathetic responses (Ayers et al., 2023). Another study examined the proficiency of ChatGPT in addressing inquiries related to common eye symptoms. The findings indicated that eight out of ten responses generated by ChatGPT received high ratings in terms of accuracy and relevance (Tsui et al., 2023). The study also highlighted that while AI applications such as ChatGPT are not rated optimal by experts at present, they hold promise for integration into clinical practice to alleviate the increasing burden and costs associated with healthcare services (Tsui et al., 2023).

Similarly, many novice runners (which lack access to professional coaches) interested in improving their endurance capacity may turn to ChatGPT seeking advice regarding training plans. However, it is currently unknown if training plans generated by ChatGPT are appropriate and in-line with recent scientific evidence, and if the AI-derived training plans differ based on provided input information granularity. Therefore, the aim of this research was to investigate the quality of running training plans generated by ChatGPT, and investigate quality differences based on provided input information.

To evaluate ChatGPT derived training plans, we followed the example of other studies performed e.g. in the medical field (Ayers et al., 2023; Lukac et al., 2023; Seth et al., 2023). We engaged experienced coaches to assess the provided training plans on the aspects outlined below on a 1 to 5 Likert Scale. Table 1 shows the rating questions and scale.

To rate the training plans, each coach had to have at least a Master’s degree in sports science, and at least 5 years of endurance coaching experience of at least Tier 2 “Trained/Developmental” athletes, as defined by a recently published framework (McKay et al., 2022). The study was approved by the Faculty’s Exercise Science and Training Ethical Committee of the University of Würzburg (EV2023/7-2609) and performed in accordance with the Declaration of Helsinki. Coaches gave their informed consent to participate in the study.

Different aspects have to be considered when assessing quality of training plans which might differ according to the underlying model or framework, the specific population and their characteristics (e.g. training status, health condition, age), sport-specific aims, and the timeframe for which the training plan is supposed to be (e.g. an individual training session, weeks, months, or years) (Jeffries et al., 2021; Morton et al., 1990; Borresen and Lambert, 2009; Mujika et al., 2018; Grosser et al., 1986; Ferrauti and Remmert, 2020; American College of Sports Medicine, 2013; Gronwald et al., 2020; Sperlich and Holmberg, 2017; Platen and Schaar, 2003).

Acknowledging these differences, the primary aspects recommended in the literature when designing training plans for novice runners include:

1. Screening for individuals at increased risk for adverse exercise-related events, such as cardiovascular, pulmonary, and metabolic related diseases, as well as other conditions (e.g., pregnancy, orthopedic injury) (American College of Sports Medicine, 2013; Platen and Schaar, 2003)
2. Definition of a goal (American College of Sports Medicine, 2013; Ferrauti and Remmert, 2020; Platen and Schaar, 2003),
3. Definition of a reliable and valid testing procedure to assess initial performance status, to derive individual training variables (e.g. heart rate at the first ventilatory/lactate threshold), and to define training effects (e.g. performance, physiological, subjective, biomechanical or cognitive measures) (American College of Sports Medicine, 2013; Ferrauti and Remmert, 2020; Currell and Jeukendrup, 2008; Platen and Schaar, 2003; Jeffries et al., 2021),
4. Use of a reliable and valid monitoring strategy (Currell and Jeukendrup, 2008; Ferrauti et al., 2020), which may include internal load (e.g. heart rate), external load (e.g. covered distance), and/or contextual factors (e.g. environmental temperature, hypoxia) (Jeffries et al., 2021; Sperlich and Holmberg, 2017),
5. Definition of training type (e.g. high-volume training, high intensity interval training, strength training) and specific training variables including but not limited to frequency, intensity, and volume (American College of Sports Medicine, 2013; Garber et al., 2011; Ferrauti and Remmert, 2020; Platen and Schaar, 2003). Additional considerations may incorporate strategic variation of volume, intensity and frequency (i.e. type of periodization) (Mujika et al., 2018). Periodization gains importance when training is planned with increased training sessions on a long term or with a specific competitive aim,
6. The progression of training over time (American College of Sports Medicine, 2013).

Next to these primary training related aspects, secondary aspects may be considered when prescribing (evidence-informed) training plans such as:

1. Nutritional aspects (e.g. carbohydrates intake, hydration) (Achten et al., 2004; Burke et al., 2011; Kerksick et al., 2017),
2. Recovery procedures (e.g. sleep) (Walsh et al., 2021),
3. Psychological skills (e.g. motivation, pain and fatigue management) (Mujika et al., 2018), and
4. Skill acquisition aspects (e.g. running technique) (Krabak et al., 2019)

To keep a narrow research scope, in this study our main focus was on assessing primary aspect, and less emphasis was on the evaluation of secondary aspects.

As users interact with ChatGPT using chat prompts, we assume that the input provided by runners to generate a training plan will vary like any other conversation. Depending on factors such as the runner's education around training procedures or own training history, some may provide minimal information, while others may be more detailed (e.g. with details about their training status and history, goals, and time availability). To accommodate this diversity in the input information, we developed three distinct initial questions. Our goal was to create these questions based on the varying levels of knowledge that people possess regarding training. Some individuals may ask basic questions about training plans, while others more knowledgeable in this area may ask specific enquiries and provide more detailed information. As a starting point, we employed a fictional 20 year old male runner aiming to use ChatGPT to generate his running training plan.

The initial questions for ChatGPT to obtain the three training plans were:

1. Please provide me with a running training plan for the next 6 weeks.
2. I am a 20 year old male who runs 2 times a week. Each run is 8 kilometers long and takes me about 30-40 minutes to complete. I have a smartwatch. I would like to increase my running performance. Please provide me with a running training plan for the next 6 weeks.
3. I am a 20 year old male who runs 2 times a weeks since one year. Each run is 8 kilometers long and takes me about 30-40 minutes to complete. My mean heart rate during these runs is around 155-170 beats per minute. I do not do other sports and I do perform only long runs and no high intensity interval training sessions or similar. I have no health issues. My goal is to increase my running performance by 3-5% in the next 6 weeks. I have access to a breathing gas analyzer and a treadmill for performance tests. For monitoring purposes, I do have access to a smartwatch which can track my heart rate and covered distance during runs, as well as environmental temperature. Please provide me with a running training plan for the next 6 weeks.

Since users interact with ChatGPT using chat prompts, we incorporated check-backs to allow ChatGPT to improve responses to each inquiry for a training plan. These check-backs were designed to anticipate the questions that someone using ChatGPT to create training plans would ask. For instance, the question in 1) was not elaborated upon through check-backs, while multiple check-backs were permitted for 3) to fine-tune the training plan. The complete conversation with ChatGPT is available in the Appendix Table 3, Appendix Table 4 and Appendix Table 5). We used ChatGPT (Version 3.0.1) to generate training plans on May 23, 2023 without any additional usage of plug-ins to the software.

We calculated descriptive statistics for the Likert scores on all rated items for each question. To test for significant differences in all rated items between the training plans, a Friedmann Test with Bonferroni Correction was performed. Significance level was set to p < 0.05. Fleiss´ Kappa was calculated to assess inter-rater reliability (Fleiss, 1971). All statistical analysis was performed in SPSS, Version 28 (IBM, New York, USA).

A total of 10 raters (age: 33 ± 5 years; 4 with a PhD, 6 with a Master’s degree in Sports Science) with 7 ± 2 years of coaching experience in endurance-based sports participated. Raters had coached runners from Tier 2 “Trained/Developmental” (n = 4), Tier 3 “Highly Trained/National Level” (n = 2), Tier 4 “Elite/International Level” (n = 2) and Tier 5 “World Class Level” (n = 2) according to a published framework (McKay et al., 2022). For training plan 1, 2 and 3, Fleiss´ Kappa was 0.43 (p = 0.00), 0.247 (p = 0.00) and 0.00 (p = 0.00). Descriptive statistics and results for significance testing can be found in Table 2.

For the question “rate the overall training plan”, training plan 1, 2, and 3 received a median rating of 2, 3, and 4 on the 5-point Likert scale. Training plan 1 differed significantly from training plan 2 (p = 0.005) and training plan 3 (p = 0.003), while training plan 2 was non significantly (p = 1.0) different from training plan 3 for this question. For training plans 1, 2 and 3, a median rating of <3 was given 19, 11, and 1 times, a median rating of 3 was given 3, 5, and 8 times and a median rating of >3 was given 0, 6, 13 times, respectively. Training plan 1 received significantly lower ratings (p < 0.05) compared to plan 2 for 3 criteria, and 15 times significantly lower ratings compared to training plan 3. Training plan 2 received significantly lower ratings (p < 0.05) compared to plan 3 for 9 criteria (Figure 1).

Runners often lack access to evidence-informed training plans or access to well-educated coaches. With the rising availability of artificial intelligence tools such as ChatGPT, runners will therefore likely seek advice from these technologies to advice on their training program. Therefore, we evaluated three six-week ChatGPT-generated training plans for runners based on different granularity of input information. We found that the quality of training plans provided by ChatGPT differed based on the granularity of input information, with less input information resulting in lower Likert-Scale ratings compared to more input information. We showed that Training Plan 1 had significantly lower median ratings on the 5-point Likert-Scale on the question “rate the overall training plan” compared to Training Plan 2 (p = 0.005) and Training Plan 3 (p = 0.003).

Training plan 3 (most input information granularity) significantly outperformed training plan 1 (least input information granularity) on 15 out of 22 criteria (p < 0.05), and training plan 2 (medium input information granularity) outperformed training plan 1 on 9 out of 22 criteria (p < 0.05). Importantly, even the training plan with the most input information (Training Plan 3) only received a median neutral ranking (“3”) for the following criteria: health screening, testing procedures (regarding the definition of training variables and evaluation of training effects), monitoring of contextual factors, prescribed and progression of training frequency, and training of psychological skills and skill acquisition. Moreover, only a median rating of lower then neutral (below 3) was given in regard to progression of volume. These results suggest that even the herein best rated training plan can be improved and is not rated optimal by coaching experts.

The quality of the training plan was found to be dependent on the provided input information granularity. Consequently, users that provides more information receive more detailed training recommendations. Although ChatGPT produced responses, it did not ask feedback questions as a coach typically would during practice. These questions serve the purpose of obtaining additional information for evidence-based decision making, thereby refining the training plans and tailoring them to individual needs. For instance, raters noticed that in training plan 3 training variables were increased too rapidly, therefore violating the individual progression and potentially elevating the risk of running-related injuries. When experienced coaches design and monitor training plans, they typically communicate with the runner directly, asking questions about their preferences and willingness to take certain risks, or whether they desire a significant increase in training outcome for potential performance improvements, although this may raise the likelihood of injuries. By posing such questions, a coach likely establishes a more suitable training plan aligned with the athlete's objectives. Additionally, coaches may enquire about the athlete's need or preference for nutritional or recovery guidance, allowing for the provision of relevant recommendations. These questions serve as valuable guidance for novice runners and most likely enhance adaptation and performance as long as they are provided, followed, and adjusted accurately. However, due to the absence of direct enquires, the training plans provided by ChatGPT offered limited or no information pertaining to these crucial aspects. Consequently, when generating training plans with ChatGPT, the users’ knowledge to input relevant information determines the AI-system output, and this could impair its usefulness for less-educated and/or novice athletes.

Currently, the source code of ChatGPT is not publicly available and the sources (e.g. peer-reviewed articles, blogs, webpages etc.) that feed the ChatGPT algorithm are unknown. Other studies suggest that ChatGPT processes both non-academic and academic sources and that it does not differentiate between sources of information based on their level of evidence (Alser and Waisberg, 2023). The heterogenous quality level of the processed AI-information might be one reason limiting comprehensiveness or accuracy of the provided responses by ChatGPT. For example, there are numerous blogs about the benefits of applying a cool-down post-exercise and these may have been used by ChatGPT in training plan 2 to recommend active recovery as an effective recovery modality. Yet, current evidence does not support a cool-down for effective recovery (van Hooren and Peake, 2018; Wiewelhove et al., 2018). Therefore, common misconceptions that are prevalent in non-scientific articles (e.g. blogs) may also be used as advice by ChatGPT. A previously raised concern was that its responses can appear confident and convincing (Seth et al., 2023). The convincing responses may result in unconditional trust in ChatGPT generated responses, potentially causing friction in a coach-athlete relationship. To overcome this issue, it may be beneficial if ChatGPT could automatically provide both scientifically trustworthy citations and an evidence ranking for its answers (e.g. based on an evidence-pyramid) (Schünemann et al., 2003).

Currently, ChatGPT is based on information provided by literature of all sources and does not individualize training prescription except for incorporating the user’s information provided in the chat window. However, with wearable technologies such as smartwatches or smart patches collecting individual data (e.g., heart rate, blood pressure, sleep related parameter) more continuously, conveniently, with increasingly reliability and validity (Düking et al., 2020a; Sola et al., 2022; Vybornova et al., 2021; Altini and Kinnunen, 2021), and with a growing number of athletes having access to such technologies, individual information on physiological parameters is increasingly available. Arguably, the integration of such information into algorithms such as those used by ChatGPT may result in more individualized and improved training plans.

We would like to highlight that ChatGPT is just one large language model runners and coaches are confronted with. Other large language models in various stages of development include BioGPT (Massachusetts Institute of Technology, Boston, MA, USA), Google Bard (Google, Mountainview, CA, USA), Sparrow (Deepmind AI, London, UK), Pangu Alpha (Huawei, Shenzen, China), and Megataron Turing MLG (Nvidia, Santa Clara, CA, USA) (Li et al., 2023). Given the development and availability of these technologies and its fast adoption rate, we assume this type of technology will, at least in some form, be used for generating training plans. To benefit from the technological advancement around artificial intelligence while ensuring scientific trustworthiness to optimally enhance runners' health and performance, runners must be educated in the strengths, weaknesses, opportunities, and threats of AI for training plans prescription, and must have knowledge around exercise and training to implement such technologies beneficially.

This study evaluated three different training plans generated by ChatGPT based on varying levels of input information granularity. This procedure allowed for a comprehensive assessment on how training plans quality differed based on the amount of information provided. Moreover, training plans were evaluated by coaching experts who possessed well-educated backgrounds and extensive experience in the field.

This study is limited to the ChatGPT version on May 23, 2023. Due to the rapid advancements in this domain, it is possible that newer iterations of ChatGPT may yield more precise outcomes for training plans and should be investigated. The interrater reliability exhibited a decrease from Training Plan 1 to Training Plans 2 and 3, despite the raters possessing well-educated backgrounds and extensive experience. This decline in interrater reliability can potentially be attributed to the absence of a universally accepted and evidence-informed consensus regarding the criteria defining an optimal training plan (e.g. (Foster et al., 2022; Burnley et al., 2022). Additionally, the individual coaching style of each rater plays a significant role in this context. For instance, empirical evidence suggests that a cautious approach to training progression is advisable to mitigate the risk of injuries. Nevertheless, there remains a lack of evidence-informed consensus on the precise definition of a “too rapid” progression and the practical methodologies for calculating load progression (Schwellnus et al., 2016; Soligard et al., 2016; Impellizzeri et al., 2020). The determination of load progression is further influenced by the unique coaching styles and the athlete’s training status, and it usually originates from a coach-athlete discussion. It is imperative to acknowledge that the notably poor interrater reliability observed in the assessment of Training Plan 3 represents a limitation in the present analysis. This emphasizes the importance of exercising caution when applying training plans generated by ChatGPT, particularly for novice runners, to minimize potential adverse health outcomes. In practice the involvement of multiple raters, such as experienced coaches, in the evaluation of ChatGPT derived training plans is advisable to enhance their quality and safety.

Future investigations should focus on evaluating the effect of training plans generated by ChatGPT (or similar AI systems) compared to traditionally formulated plans by certified coaches of different levels. Additionally, further research is required to examine the interaction effects between coaches and ChatGPT (or similar AI) to address queries regarding time efficiency in generating training plans and related aspects. In addition to designing training plans, there are other important factors that runners should consider if they want to improve their health and/or performance. These factors include motivation, training monitoring, and frequent adjustments to the plan, and other aspects that are typically handled by coaches. ChatGPT is currently not able to provide assistance with these factors and consequently cannot fully replace coaches.

We showed that the quality of training plans for novice runners generated by ChatGPT is dependent on the provided input information granularity, and consequently on the user’s knowledge about planning of running training. Importantly, even the best performing training program included suggestions that are not rated optimal and lack evidence-informed planning, demonstrating potential for improvement. Based on our results, we cannot recommend to employ ChatGPT generate training plans in practice for runners, without checking the correctness of provided recommendations. Nevertheless, it may assist in designing training plans for well-informed individuals.