This study examined the accuracy of a newly developed video-based kinematics computation system, namely VCS, in instantly providing objective and comprehensive assessments of young children’s FMS under the TGMD-2 framework. Unlike the conventional TGMD-2 rating approach, which depends solely on humans’ visual analysis, the key parameters for determining assessment scores in the proposed VCS are based on data captured by the Kinect v2 sensor and the VCS. Through computing a series of specific parameters, such as joint angles and duration off the ground, the system can quantify the score for each performance criterion according to the TGMD-2 locomotor subtest guidelines. By incorporating the marker-less Kinect v2 sensor, the VCS can be an assistive tool for an inexperienced rater to evaluate children’s locomotor skills in an instant, without the necessity for the presence of an experienced rater. At the same time, it yielded comparable accuracy with the conventional TGMD-2 assessment approach. In terms of efficiency, the VCS can provide accurate and objective assessments in a short time. According to the TGMD-2 guidelines, an experienced and well-trained rater is required to assess each locomotor skill in real-time. However, it is formidable for a rater with less experience to evaluate each skill simultaneously as they are required to observe and identify whether the child has achieved the performance criteria for that skill or not. Alternatively, videos can be recorded for the inexperienced rater to confirm the ratings, which can be particularly useful for ambiguous cases when skill performance can be reviewed repeatedly, in slow motion. However, this is a process that involves substantial time and human intervention. A study also concluded that even though the raters may be well-trained and experienced, the approach is confined in terms of the ability to identify all performance criteria during real-time assessment (Ward, 2019). The results of this study showed that the VCS was a potential alternative to rating children’s FMS performance, particularly in contexts where there are no experienced raters available, for example in kindergarten settings. This system can thus serve as an alternative to the conventional approach, especially for inexperienced raters, since it facilitates kinematic data processing and sufficiently reduces the necessity of human intervention in larger-scale assessments of FMS. In general, the research yielded identical results for the mean, mode and median of the threshold of each performance criterion (Table 5) after 1000 times’ random sampling and comparison, suggesting uniform distributions of the thresholds. This evidences that the thresholds for each performance criterion obtained would be the optimal results. Favourable agreement levels were discovered between the VCS and the rater who conducted visual analysis, thus revealing that the VCS can achieve accurate and comprehensive assessments with low latency. The mean percentage agreements between the VCS and the rater in terms of the scores for each skill ranged from 66.1% to 87.5%. This finding suggests that the VCS can be used to identify most distinctive features and even minor details associated with all six skill motions, similar to the human rater’s performance. However, some performance criteria were associated with percentage agreement values of less than 60%; for example, performance criterion 2 and 5 under hop were associated with percentage agreement values of 43.6% and 55.7%, respectively. The decrease in agreement may be ascribed to the field of view of the Kinect v2 sensor in a single side of the runway (Webster and Celik, 2014). Consequently, the view of children’s execution of the skills was restricted to the limited capture range of the sensor and the joint movements on the other side were occasionally blocked from occlusion and unable to be captured. Besides, the use of a non-preferred leg would hinder children from successfully hopping. Since hopping requires tremendous muscular strengths and balancing skills (Krasnow and Wilmerding-Pett, 2015), the hopping techniques were still improving for children aged above five (Krasnow and Wilmerding-Pett, 2015). Hence, hopping with non-dominant leg would be difficult for early aged children. The paired t-test showed that there was no significant difference but a significant correlation between the standard scores rated by the VCS and the trained rater. From Tukey mean difference plot analysis, the two rating approaches did not differ significantly in terms of the standard score at the group level. The mean difference of the two approaches was close to zero difference, suggesting there was no bias. However, the 95% confidence interval limit of agreement indicated considerable variations at the individual level that the VCS approach may be 3.71 scores inferior or 3.38 superior to the rater approach. These results reveal that the VCS had certain levels of agreement with the traditional rating approach at the group level, whereas the agreement between two approaches at the individual level would require further improvement. Regarding descriptive ratings, the difference between the VCS and the rater’s mean scores was not significant, demonstrating that the VCS could be used for objectively and comprehensively assessing children’s FMS. The percentage agreement and kappa agreement between the VCS and the rater in terms of their descriptive ratings were found to be good and moderate, respectively (Sasyniuk et al., 2007; Cohen, 1960). However, the VCS was more likely to assign lower standard scores than the human rater, thus explaining the observation that the VCS rated a higher percentage of children as “poor” and “very poor” compared with the human rater. In addition, it was noted that there were no participants rated “very superior” or “superior” by either the rater or the VCS, which was possibly an indication of general inferiority of locomotor skills in the samples. This was one of the limitations pertained to the insufficient sample age range on account of convenience sampling. In this study, children aged nine and 10 years were not included, despite them being within the proper age range based on the TGMD-2 guidelines. Therefore, future evidence is necessary to ascertain whether the VCS can classify children with good FMS ability within the categories of “superior” or “very superior”. By developing the VCS, this study contributes to the current understanding of approaches for designing algorithms to instantly assess children’s FMS without the presence of an experienced rater. Similarly, a previous study presented an approach that involved the use of wearable sensors to assess 23 of the 24 criteria under the TGMD-2 locomotor subtest (Bisi et al., 2017). This approach was validated as having reliably and objectively evaluated FMS (Bisi et al., 2017). However, the approach could not provide a rapid or reliable assessment of the performance criterion 3 pertaining to running. The approach not only involved a time-consuming process of placing several sensors on each participant’s body but also required two minutes to compute the results for each participant. Accordingly, this approach does not constitute a fully automated system in that it lacks time efficiency because of its relatively long running time and its requirement of manually evaluating performance criterion 3. In contrast, for performance criteria and descriptive ratings, the VCS proved to be objective, accurate, and easy to use due to the incorporation of marker-less devices. Furthermore, the accuracy of the VCS achieved in the current study (72.4%) was comparable to that obtained by using wearable sensors (73%) (Bisi et al., 2017). Therefore, moving to a marker-less system is warranted in that the VCS obviates the recording of video clips, extensive rater training, and additional system running time. It substantially simplifies the assessment procedure and saves considerable time. Apart from the insufficient sample range, another limitation that might arise was evident when the Kinect v2 sensor was used in the VCS. Considering the study findings, the accuracy between the VCS and the rater would be adversely affected by the sophisticated threshold determination processes and limited field of view of the Kinect v2 sensor. The Kinect v2 sensor was unable to track and distinguish different body joints when overlapping of joints occurred, thereby leading to a decrease in accuracy. A study has documented the possibility of the use of two Kinect sensors on either side to collect FMS data by converting the joint positions of one sensor into the world coordinate system of another (Rosenberg et al., 2016). To address the limitation of the Kinect v2 sensor, future studies should consider using more than one sensor to capture movements in order to collect kinematic data from both sides of the runway. This has been a preliminary study developing and evaluating the accuracy of the VCS in assessing young children without disability or other health issues. However, existing empirical studies have validated TGMD-2 for children with health problems (Houwen et al., 2010; MacDonald et al., 2013; Pan et al., 2009). Hence, children with wider range of ability, for example, diagnosed with developmental delays, could be included in experimental assessments in future clinical research in order to improve and validate the general applicability of the VCS. A further validation study should be conducted to see whether the system is sensitive to differentiate children with developmental delay from typically developed children. For the sake of early screening for developmental delay, more subject recruitment in different ages would be required to establish a local reference norm. Moreover, the proposed system was verified to be feasible for performing video-based skill assessments by using the object control subtest of TGMD-2 as well as of the new version, TGMD-3 (Allen et al., 2017). Details of the spatiotemporal and kinematic parameters used in this study can be accessed for further analysis of the FMS of the participants in this study. The present results may serve as a valuable resource for intervention and rehabilitation studies. Future studies on the automation of VCS under the TGMD-2 standard should also investigate the possibility of using portable devices (e.g., smartphones or tablets) other than Kinect v2 sensors to further enhance the applicability and convenience of systems for motor control monitoring. Direct and more informative feedback could be provided with the help of a user-friendly app. For achieving accurate and time-efficient predictions based on kinematic computations, the VCS could be improved to identify key parameters related to additional activities under other physical evaluation frameworks, such as the Movement Assessment Battery for Children, Second Edition (Henderson et al., 2007), and the Bruininks–Oseretsky Test of Motor Proficiency, Second Edition (Bruininks and Bruininks, 2005). Doing so can enable comprehensive, efficient, and objective evaluation of the full scope of children’s development (Logan et al., 2017). |