Introduction
Fundamental movement skills (FMS) such as running, catching and jumping are considered the foundations of physical activity (Gallahue et al., 2012). Once a child has developed FMS mastery he/she theoretically should be able to progress to developing general motor literacy and potentially sports specific skills (Clark and Metcalfe, 2002). For instance, once a child has mastered a ‘kick’ he/she can then develop specific types of kicks used in different sports (e.g. a ‘punt’ kick in Australian Football, or a ‘soccer’ kick in Football). Likewise, the movement mastered in an overhand throw is integral to perfecting a tennis serve or throwing to a cut-off in baseball or softball. In this sense, FMS can be termed ‘generic’ movement skills with the possibility that these can be subsequently fine-tuned for specific sports applications (Clark and Metcalfe, 2002; Gallahue et al., 2012; Seefeldt, 1980).
When aiming to assess FMS improvement, a combination of object control (involving the reception or propulsion of an object with either the hand or foot), locomotor (travelling from one point to another) and balance skills (e.g. twisting, turning, and bending) are generally assessed (Henderson et al., 2007; New South Wales Department of Education and Training, 2005; Ulrich, 1985). Process-oriented instruments (e.g. Test of Gross Motor Development-2, TGMD-2 (Ulrich, 2000)) assess the components or critical elements of each skill considered important to master the skill rather than the outcome of the skill execution itself (such as time, distance, accuracy, or number of successful attempts) (Burton and Miller, 1998). Process-oriented instruments are typically used when evaluating FMS (Morgan et al., 2013) as they are more effective in identifying skill deficits, which allows for identification of skill critical elements to focus on for an outcome improvement. Process-oriented assessment of FMS can be characterised by the use of a series of these critical elements, recorded by an assessor as either ‘achieved’ or ‘not achieved’. Therefore process-oriented measures may be more salient for child skill assessments as the information can be used to inform the teacher or coach which specific skill components need practice and therefore they can give specific ideas for improvement (Hands, 2002). Moreover there is some evidence that process and product movement skill assessment scores in children are not highly correlated (Logan et al., 2011), showing that to some degree they are assessing different things, although the literature in this area is limited (Stodden et al., 2008).
A critique of this assessment type is that the ‘either/or’ dichotomy may not allow for children’s progression in their skill development, with floor and ceiling effects also a risk (Gallahue et al., 2012). For example, the strike in the TGMD-2 includes the component: ‘Steps with non-preferred foot’; marked as ‘achieved’ or ‘not achieved’. Whereas, if using a developmental sequence approach to assessment, four levels of ‘stepping’ could be distinguished in a strike (i.e., 1. No forward step, 2. Forward step with ipsilateral leg, 3. Forward step with the contralateral leg, and 4. Long forward step with the contralateral leg (Langendorfer, 1987). Rating the child using a dichotomous marking system would mean the child would be classified as either ‘level 1’, or any of the next three levels, resulting in the test only being able to discriminate at a low level. Nevertheless, if one of the assessment aims is to discriminate between low and somewhat moderately skilled individuals, the use of such process-oriented instruments are well justified.
Effort is occurring at a national level in certain countries to encourage children into golf. For example, in America, The First Tee, a non-profit USA organization (Program descriptions http://www.thefirsttee.org Accessed on 22nd August 2014) provides guidance to schools, communities, and parents in ways to increase young children’s golf exposure. Similarly, Australia has MYGolf!, a National junior golf program (http://www.mygolf.org.au/, Accessed 7th August 2014). Yet in order to evaluate early level golfing proficiency, a basic or fundamental process-oriented assessment is needed to identify skill deficits and diagnose errors (Winnick, 2009). Coaches, teachers, and parents could use assessment results to provide feedback that aligns with specific errors that the young golfer needs to improve. However, as coaching of young golfers may be delivered by individuals from a range of backgrounds, process-oriented assessments need to be simple enough to accommodate coaches, teachers, and parents whose golf knowledge varies from novice to expert.
While there are instruments which assess a range of FMS, there are no published reliable and valid process-oriented skill assessments to assess children’s golf striking ability. A recent systematic review of tests examining skill outcomes in sport, (Robertson et al., 2014) highlighted only three golf tests with documented reliability and/or validity measures, none of which have been tested in children. Additionally, these tests could all be considered product assessments (outcome score), with the first (unnamed) designed to assess putting and pitching and tested in adult males (Porter et al., 2007) and the second and third (the ‘Nine-ball skills test’ and ‘Approach-iron skills test) both reported using elite and sub-elite adult males (Robertson et al., 2012; 2013).
The First Tee has developed sample product-oriented assessments for children (i.e. a test for a 9-10 year old can include whether a child can hit the fairway every 2 out of 5 shots with a wood, hit the green every 2 out of 5 shots with an iron and complete 9 holes under two hours, (http://www.thefirsttee.org). Whilst immediate knowledge of results (Schmidt and Wrisberg, 2008) can be motivating to players (Deci and Ryan, 2008), these product tests provide limited diagnostic information for stakeholders interested in improvement of young golfers. Process-oriented evaluations targeted towards physical education teachers typically informally evolve from the development of a list of critical elements (Graham et al., 2013; Hopple, 1985; Rink, 2009). Thus, given the rising interest in youth golf participation, the purpose of this study was to develop an age-appropriate, process-oriented assessment for children’s golf strike and putt strokes based on similar FMS assessment protocols.
Methods
Instrument development
Rather than develop a new assessment approach potentially not compatible with established assessment models, we adopted the TGMD-2 format; a common process-oriented assessment of FMS competency in children (Ulrich, 2000). The TGMD-2 assesses FMS competency in 12 skills (one of which is a strike) and has been validated for use in 3 to 10 year old children (Ulrich, 2000). Briefly, the TGMD-2 assessment protocol includes demonstrating each skill to the child prior to the child completing two trials for each FMS (Ulrich, 2000). The number of components performed correctly is summed to produce a total score for each skill, and these can be summed for a total FMS score.
The fundamental skill used in golf is a strike, therefore the TGMD-2 provided a starting point in terms of the strike assessment. The TGMD-2 assesses ‘Striking a Stationary Ball’ using a baseball bat and the skill components that are assessed are in line with this type of strike. For example, component 2 of the TGMD-2 strike is: ‘Non-preferred side of body faces the imaginary tosser with feet parallel.’ The third version of the TGMD (due for release in 2015) also includes this assessment with some slight changes. In addition, the TGMD-3 will assess the ‘One Hand Forehand Strike of a Self- bounced Ball’ as it was recognised by the developer that this strike is popular in many sports around the world (e.g. tennis, badminton) (Personal Communication, Professor Dale Ulrich 27th November 2013). However, the one-hand forehand strike assessment does not include component 2 of the TGMD-2 two-handed side strike as it would not be appropriate to the one-hand side strike (i.e. the non-preferred hip/shoulder is not required to be facing straight ahead for a one hand forehand strike). Thus, although both forms are considered fundamental strikes, they can both be considered context-specific as they are evaluated in a manner according to the form and sport for which they are associated. Therefore, to evaluate the two most common strikes undertaken in golf, the fairway swing and putt, context specific fundamental assessments were developed using the TGMD 2 and 3 assessment formats as the guide.
Delphi
Convening an expert panel is a common method used to guide the development of sports skill assessments (Robertson et al., 2014) and can be used to determine content validity of the assessment; whether a test is a measure of skills relevant to a particular activity (Barrow et al., 1989; Sireci, 1998). Participants in the expert panel were sought via author contacts and comprised of two groups, i) 13 researchers (including Authors 1, 3, 4) and ii) six golfing experts/professionals identified and compiled from a governing body for the sport in Australia ’Golf Australia’ (Research and Development -Author 2, High Performance Director, Technical Director, Golf Development Director, Junior Development Manager and the Tournament Preparation Consultant). Inclusion criteria for researchers were to have published internationally and/or be currently conducting a PhD in the area of children’s movement skill assessment. The golfing experience of the researchers in terms of current playing, coaching others and being personally coached is presented in Table 1. Golfing experts needed to fulfil at a minimum the highest criterion levels shown in Table 1.
A Delphi process was used to refine components. Researchers were initially individually emailed a preliminary test design document to comment on whilst the Golf Australia panel contributed to the same document via a group discussion. The initial document outlined the aim to develop a process-oriented assessment for both the golf swing and the golf putt with the intention that these be in line with the TGMD (in terms of having a number of components that are achieved or not achieved over two trials). The main purpose of the new assessments was described as ‘to see if children have the core components (critical elements) needed to perform the respective skills to a basic level’. A copy of the TGMD-2 strike assessment and the proposed TGMD-3 strike assessments were presented for background. Following this, recommended components to be assessed within each of the golf skills drafted by Author 1 (based on experience using the TGMD-2), and Author 2 (based on experience with golf assessment) were specified for critique as appropriate. Feedback was also requested on draft instructions for the skill execution. Table 2 shows initial draft instructions and skill components for each skill.
Feedback from both researchers and golf coaches on the golf swing and putt was collated and is presented in Table 3. In general, panel members who recommended the most changes were those with both research and golf experience. Those researchers without golf experience contributed few minor recommendations. Table 3 also lists potential concerns with each skill component and how these concerns were addressed. Accordingly, Authors 1 and 2 drafted a new version based on this feedback which was redistributed to all researchers individually and back to the golf expert panel to assess any further changes.
The critique at this next point relevant to both assessments were clarifying modified equipment would be used; specifying ball placement, specifying a basic grip, and changing the order of the hit and follow through. One comment was that it may be difficult to assess reliably in the field whether a participant’s eyes were focused on the ball throughout the whole skill. Changes specific to the swing included: clarifying the height that the club reaches in the swing and that after follow through, the weight transfer is reflected by weight seen on the front foot.
Changes specific to the putt included: specifying in instructions that the ball is to land near the target, specifying the head is ‘still’ in the putt (rather than ‘relatively still’), reducing the 30 degree leeway for the ball direction to 15 – 20 degrees and clarifying a pendulum movement in the follow through. All suggestions were accommodated. The grip was changed for the swing, but not the putt, as the golf panel suggested the putt can be performed well with a range of grips. Instead of adding ball placement as a component to be assessed (as this would have meant three critical features were being scored for the ‘start’ component) it was decided the assessor would place the ball leaving the child to determine their position. Also, the degrees specifying direction were replaced with clock terminology (i.e. 11pm to 1pm) for simplicity.
Reliability with gold standard
Authors (1 and 2) then rated five videos of children performing the golf swing and five videos of children performing the golf putt. In keeping with study scope, these were videos of beginner, not elite golfers. One rater experienced in using the TGMD-2 (Barnett et al., 2014) (but with no golfing experience) was then trained (2 hours) in assessing the newly developed golf skills. This rater independently rated the five videos of the golf swing and five videos of the golf putt to assess reliability with the ‘gold standard’. Percent agreement (number of components in agreement/total number of assessable components x 100) was 88% (swing) and 78% (putt). Whilst this is reasonable, each case of disagreement (except one) was with component 2 (C2; (putt 33%, swing 50%) or component 3 (C3; putt 50%, swing 75%). Assessing ‘a straight back’ as part of C2 in each skill was problematic (e.g. are slight rounded shoulders or hyper back flexion permissible?) as was assessing whether the eyes were on the ball as indicated by the head remaining ‘still’ (putt) or ‘relatively still’ (swing). As a result these components were further revised to improve reliability. Tables 4 and 5 present the revised final version of the skill assessments with all changes underlined.
Sample
A total of 43 children from one school aged from 6-10 years (M= 7.8 years, SD= 1.3) were assessed using the revised format. This age range was chosen as it can be considered the first stage at which golf could be introduced and is also within that validated by the TGMD (ages 3-10). Parents consented and ethics approval was gained (school governing body and the University). No child had prior experience (organised lessons) in golf. Children were shown a physical demonstration of each skill. Each participant had the opportunity for a practice trial and were then assessed twice for each skill by Author 2 as per the TGMD-2 protocol (Ulrich, 2000). Scores for both skills were summed for each child resulting in a potential score range of 0-24.
Discriminate validity and reliability (intra-rater and test retest)
A one-way ANOVA was used to test whether skill competence differed by age (11 children aged 6 years), (15 children aged 7/8 years) and (17 children aged 9/10 years). Where relevant, Tukey’s post-hoc comparisons were undertaken to assess where any identified differences lay.
Intra rater reliability, the level of agreement of a single observer on multiple assessments/scoring (Streiner and Norman, 2008), was assessed on 65% of children (n = 28). These children had consent to being video recorded during the assessment with the same rater assessing the videos two weeks after the live assessment. A one way random effect (when people effects are random) ICC (1, k) was conducted to assess intra-rater reliability between the live and video assessment for the sum of the golf skills. A paired sample t-test assessed any systematic differences between assessments. Nineteen of the children were also assessed for test retest reliability using a one way random effects model at two separate times (5 days apart) six weeks after the initial test. Smaller numbers were due to end of term absences. ICC values <0.4 were rated as poor, ≥0.4 to <0.8 as moderate and ≥0.8 as excellent (Helmerhorst et al., 2012).
Results
There was a normal distribution with no child scoring close to the minimum or maximum score (range 6-21). Skill competence differed significantly across the three age groups, (F(2, 40) = 11.18, p < 0.001). Tukey post-hoc comparisons of the mean total scores across three age groups indicated that 6 year olds (M = 10.36, 95% CI [8.60, 12.12]) were significantly less skilled than 7/8 year olds (M = 13.73, 95% CI [11.98, 15.49]), p = 0.013, and, 9/10 year olds (M = 15.53, 95% CI [14.18, 16.88]), p < 0.001. The 7/8 year olds (M = 13.73, 95% CI [11.98, 15.49]) showed no skill differences when compared to 9/10 year olds, p = 0.185. The ICC for intra-rater agreement indicated close to excellent agreement (ICC = 0.79, 95% CI 0.59, 0.90) (Nunnely and Bernstein, 1994, Helmerhorst et al., 2012). The mean total golf score did not significantly differ between intra-rater assessments (M live assessment = 13.32, SD = 3.36, M video assessment = 13.14, SD = 2.69, p = 0.668). Test retest reliability on 19 children indicated moderate stability (ICC = 0.60, 95% CI 0.23, 0.82). The mean total golf score was significantly higher on the five day retest (M difference 1.42, SD 2.46, p = 0.021).
Discussion
The purpose of this paper was to describe the development of an age-appropriate process-oriented assessment instrument for children’s golf swing and putt strokes. A Delphi process was used to develop assessment components for each skill based on the TGMD assessment protocols. The final instrument did not have floor and ceiling effects and appeared to discriminate based on age, reflecting developmental skill acquisition. In other words, as children become older they generally performed better on the test which is what we would expect to see if the instrument had discriminant validity by age. Whilst the mean difference between 7/8 year old and 9/10 years olds was not statistically significant, the difference of 1.8 points suggests a larger sample may have greater power to detect a statistical difference. This age difference pattern also reflects the TGMD-2, where the increase in object control raw score means is steeper in the younger years (ages 3 to 6), starts to plateau in the middle years (ages 7/8) and flattens more in the older years (ages 9/10) (Ulrich, 2000). Importantly, the assessment was reliable when compared with video assessment, indicating the instrument can be used in the field to assess children live without the need for video analysis. We also found that the instrument had moderate stability over time, although there was a significant increase in performance equivalent to a child doing approximately one more component on one trial for one skill. This could have been the result of the tests being five days apart perhaps contributing to a slight learning effect. Constraints in the school setting limited our timing and sample size for this aspect. This testing was performed in the final week of school term and some students were absent, thereby limiting the sample size for this testing.
The study strengths are the rigorous Delphi consultation process used in the instrument development drawing upon the expertise of golf industry professionals and movement skill researchers, the ability to assess intra-rater reliability using video analysis, and the reasonable size sample comprising boys and girls. Future research could investigate inter-rater agreement in field testing as this is important when assessing the usefulness of an instrument (Barnett et al., 2009; 2015; Kottner et al., 2011). While the assessor was an experienced golf coach, future research should endeavour to see the level of training needed to achieve inter-rater agreement for an assessor with little golf knowledge (for example a physical education teacher who wishes to introduce a golf module). That we were able to train a rater with no golf experience and this person achieved reliability with the gold standard holds promise for the current instrument to be adapted across a range of settings by people with no golf background.