The Test of Gross Motor Development 2 (TGMD-2) is currently the standard approach for assessing fundamental movement skills (FMS), including locomotor and object control skills. However, its extensive application is restricted by its low efficiency and requirement of expert training for large-scale evaluations. This study evaluated the accuracy of a newly-developed video-based classification system (VCS) with a marker-less sensor to assess children’s locomotor skills. A total of 203 typically-developing children aged three to eight years executed six locomotor skills, following the TGMD-2 guidelines. A Kinect v2 sensor was used to capture their activities, and videos were recorded for further evaluation by a trained rater. A series of computational-kinematic-based algorithms was developed for instant performance rating. The VCS exhibited moderate-to-very good levels of agreement with the rater, ranging from 66.1% to 87.5%, for each skill, and 72.4% for descriptive ratings. Paired t-test revealed that there were no significant differences, but significant positive correlation, between the standard scores determined by the two approaches. Tukey mean difference plot suggested there was no bias, with a mean difference (SD) of -0.16 (1.8) and respective 95% confidence interval of 3.5. The kappa agreement for the descriptive ratings between the two approaches was found to be moderate (k = 0.54, p < 0.01). Overall, the results suggest the VCS could potentially be an alternative to the conventional TGMD-2 assessment approach for assessing children’s locomotor skills without the necessity of the presence of an experienced rater for the administration.