The kinematic analysis of sports and clinical movements provides useful information to athletes and coaches for evaluating the technical performance during competitions and training sessions. In every sport, including swimming and other aquatic disciplines, this information can be used to optimize training activities (Ito and Okuno, 2010; Pogalin et al., 2007; Slawson et al., 2010). Kinematic analysis requires excellent accuracy and robustness of the methods used for data collection, as even little variations in kinematic indices can be important. Thus, there is a great interest in developing measuring techniques for a highly accurate and sensitive analysis of human movements.

The majority of the methods for kinematic analysis are based on markers, attached or fixed to the human body that enable tracking of specific anatomical landmarks. In dry conditions, passive markers are commonly used, consisting of discs or spheres of different sizes covered by retro-reflective tape (Berthouze and Mayston, 2011; Cappozzo et al., 1995; Davis et al., 1991; Frigo et al., 1998; Knuesel et al., 2005). Recently, Kjendlie and Olstad, 2012 evaluated an automatic motion capture system designed to analyze human swimming using spherical markers with a diameter of 19 mm, reporting a 7% to 10% increase in the passive drag due to the resistance exerted by the 24 markers attached to the swimmer. Although further data are needed to support their findings, the formers may lead to the conclusion that the use of markers of non-negligible volume in the water is questionable. Indeed, an increased passive drag could negatively affect the performance of the swimmer.

A possible approach to avoid increment of passive drag consists in replacing the spherical markers with bi-adhesives placed on the swimsuit, or with non-reflective markers drawn on the swimmer’s skin (Ceccon et al., 2013; McCabe et al., 2011; McCabe and Sanders, 2012). When this setup is used, movements are filmed through conventional underwater cameras, and the resulting video recordings are analyzed using specific software for tracking of features. Manual tracking represents the roughest solution to analyze movements performed in the water. However, this tracking method requires an extensive amount of time. For example, Psycharakis and Sanders, 2008 used manual digitation to analyze 19 anatomical landmarks for 4 stroke cycles in 10 swimmers performing a 200-m front-crawl trial. The mean stroke frequency was 0.74 Hz, involving a total duration of approximately 5.4 s for the examined fraction (4 cycles) of each swim. Six cameras at 50 frames per second were used, therefore, about 1620 frames digitized for each swimmer. Although not all the markers had to be digitized in each frame (some were not visible by one or more cameras), a well-trained operator would have reasonably used no less than one minute per frame, involving a total digitation time of 27 hours for each swimmer. Therefore, the availability of appropriate software for automatic tracking would represent a significant advance in the practical use of kinematic analysis in swimming and other aquatic sports. This approach would provide a quick feedback on the kinematic characteristics of swimming to swimmers and coaches.

Several commercial computer software are available to track the markers, measure the kinematic variables, and present the data intuitively to coaches and athletes (Barris and Button, 2008). Although many of these software have tools to perform automatic tracking of markers, little is known about their algorithms and techniques of analysis. The awareness of this information is essential to optimize the automatic tracking procedure in every environments and conditions. While the process of tracking has been thoroughly established already, it is not necessarily easy to select features that can be tracked properly (Shi and Tomasi, 1994), therefore a wide variety of algorithms has been proposed that aim at developing the most robust algorithm with less computational time.

Common tracking methods are the “point tracking”, based on the detection of points that represents objects in consecutive frames, and the “kernel tracking”, representing the objects as primitive shapes and computing their motion from frame to frame (Yilmaz et al., 2006). To the authors’ knowledge, only Figueroa et al., 2003 described in detail the algorithms used in a software program (DVideo) designed for automatic tracking of markers in human motion analysis. In that software, based on the kernel tracking, the motion is computed by template matching where a similarity measure, e.g. cross correlation, is used to search for the object template in each image. While this kind of approach has a number of advantages for the analysis of sports gestures under normal visibility conditions (Figueroa et al., 2003), it may not prove to be as much effective when dealing with underwater images. The analysis of video recordings performed in an underwater environment involves additional difficulties linked to the small target size, the low image quality and the presence of background clutters. With high probability, these aspects make underwater tracking process harder.

An alternative tracking approach, that could prove to be useful in handling the typical difficulties of underwater motion analysis, is represented by optical flow techniques, such as the popular Kanade-Lucas-Tomasi (KLT) tracking (Lucas and Kanade, 1981; Tomasi and Kanade, 1991). Optical flow is a flexible representation of visual motion that is particularly suitable for computers analysing digital images. The algorithm explicitly optimizes the tracking performance by classifying a feature as appropriate if it can be tracked successfully. This technique is reported to be the most efficient and accurate among optical flow techniques in terms of average angular deviations from the correct space-time orientation (Barron et al., 1994). The method based on optical flow is complex, but it can detect the motion accurately even without knowing the background (Lu et al., 2007). In this context KLT had proven to be accurate and efficient in computing optical flow (Barron et al., 1994; Galvin et al., 1998; Liu et al., 1996;Lu et al., 2007). Barron et al., 1994 compared the accuracy of different optical flow techniques on both real and synthetic image sequences, and found the KLT the most reliable one. His findings were confirmed by Liu et al., 1996 who analyzed the accuracy and the efficiency trade-offs in various optical flow algorithms, and showed that KLT presents better accuracy with reduced computation time. Finally, Galvin et al., 1998 evaluated eight optical flow algorithms and concluded that the KLT method consistently produces accurate depth maps with a low computational time, showing good tolerance to the presence of noise. More recently, this technique has proven to be the most efficient in automatically estimating vehicle speed from video sequences acquired with a fixed mounted camera for its robustness in presence of noise (Shukla and Patel, 2013). Therefore, our hypothesis was that the KLT feature tracker could be the most appropriate technique for tracking underwater images.

The aim of the present study was to evaluate the effectiveness of a software for automatic tracking of user-defined features in underwater conditions. The software has been developed starting from a free open-source implementation of the KLT feature tracker (Sinha et al., 2011), and was already used in a previous study that analysed the front crawl swimming (Ceccon et al., 2013). It is recognized that an important characteristic of an effective tracking algorithm is a limited amount of manual interventions required throughout the tracking process (Chiari et al., 2005; Figueroa et al., 2003). The percentage of manual interventions necessary to supervise the automatic tracking was, therefore, chosen as the criterion to evaluate the algorithm.

Automatic tracking of underwater movements was performed by a developed (DVP) and commercially available software (COM, SIMI Reality Motion Systems GmbH, version 7.5.288). To our knowledge, there is no commercial automatic tracking software available with known characteristics and precision. Therefore, COM was chosen as the reference software based on the consideration that it is one of the most widespread software in human movement research.

The algorithm has been specifically developed to perform the automatic tracking of passive markers, providing a simple user interface to Birchfield’s implementation of the KLT tracker (Birchfield, 1997). Given an initial position of a marker, set by an experienced operator, the automatic tracker uses the gradient of the spatial intensity to optimally search for the vector that minimizes the difference between the surroundings of the feature in adjacent frames, as proposed by Lucas and Kanade, 1981 and developed by Tomasi and Kanade, 1991. In brief, the algorithm aims to find the displacement of small patches between two consecutive images. Under brightness constancy assumption, the changes in the image intensity are only due to motion. Thus, the intensity of a patch should correspond to the intensity of the patch in the following image. This correspondence can be expressed in function of the displacement that has occurred on the image plane. The value of the displacement is calculated by minimizing the residual error, after having approximated the intensity function by its Taylor series expansion truncated to the linear term. The minimization step is repeated iteratively in a Newton-Raphson style, upon registration of the image patches according to the current solution (Lucas and Kanade, 1981). The algorithm used here has the advantage of being easily implementable starting from the KLT algorithm open source codes, available at the Intel Open Source Computer Vision Library (http://software.intel.com/en-us/articles/intel-integrated-performance-primitives-intel-ipp-open- source-computer-vision-library-opencv-faq).

A sample of 21 AVI video files (720 x 572 pixels, recorded at 50 framesâ‹…s^-1) was used to perform both manual and automatic tracking of sports underwater movements. The videos were taken from preliminary studies in which each participant had signed an informed consensus in accordance with the local institutional board. Conventional underwater analogic cameras (Sony Hyper Had, TS-6021PSC, Japan) were connected to a computer through an Analogic to Digital Video Converter (Canopus ADVC55, Grass Valley, USA). The cameras were automatically synchronized with an ad-hoc software application (Ceseracciu et al., 2011). In regards to the geometric distortion correction, the calibration of the cameras’ intrinsic parameters was achieved from a dry land acquisition of a checkerboard pattern that was corrected for underwater application (Lavest et al., 2003; Zhang, 1999). This kind of calibration was used because it was easier to be performed thanks to the absence of bubbles and water turbulences. Then, a correction factor of 1.333 was applied, taking into account that the acquisitions were performed with the same cameras but in underwater conditions (Lavest et al., 2003). Videos corrected from geometric distortion were used as input for the analysis. The recorded sequences were de-interlaced before the tracking procedure.

Four different types of markers were used, with black-on-white or white-on-black contrasts between the markers and the background: full black circles on the white skin, full white circles on the black swimsuit, black crosses on the white skin, and partially black circles (two full and two empty quarters) on the white skin (Figure 1). The markers were created by drawing them on the skin with indelible and harmless ink, or by attaching white circular bi-adhesive tapes (BTS, Milan, Italy) on the swimsuit. For each specific movement analyzed, the markers were positioned on three of the following anatomical landmarks: styloid process of the ulna (SU), olecranon of the ulna (OU), acromion of the scapula (AS), inferior angle of the scapula (IS), greater trochanter of the femur (GF), lateral epicondyle of the femur (LE), and lateral malleolus of the fibula (LM). The examined videos were divided into six sets, according to the underwater activity performed: 1) two videos of walking on a water treadmill (Figure 2A) with examined markers (full black circles) on GF, LE and LM; 2) two videos of running simulation on a water elliptical machine (Figure 2B) with examined markers (full black circles) on GF, LE and LM; 3) six videos of front crawl swimming (Figure 2C) with examined markers (partially black circles) on SU, OU and AS (Figure 2C); 4) two videos of walking in the swimming pool (Figure 2D) with examined markers (black crosses) on GF, LE and LM; 5) six videos of passive towed swimming in a prone position, using a traditional brief swimsuit (Figure 2E) (n = 3 videos) or a full body swimsuit (Figure 2F) (n = 3 videos) with examined markers (full black circles in the first case, both full black and white circles in the second case) on IS, GF and LE; and 6) three videos of flutter kick swimming (Figure 2G) with examined markers (black circles) on IS, GF and LE. One underwater camera was used to record the subject’s sagittal view in all sets of videos, with the exception of front crawl swimming, where six cameras were used as the setup for a 3D analysis. In this last case, the six cameras were positioned to maximize both the sagittal and coronal views of the swimmer’s right upper limb, as described in a previous study (Ceccon et al.,2013). Regarding the other five motor tasks, each of them was recorded with one camera only. The camera was fixed to the pool’s edge using a support 2 m apart from the exercising subject, 30 cm below the surface of the water for towed and flutter kick swimming videos, and 50 cm for walking, running and elliptical machine videos.

Each video was tracked manually by an experienced operator using the manual-tracking mode of the COM software to determine the reference for the 2D coordinates of the center of the markers. Subsequently, the 21 videos were tracked automatically by using both DVP and COM. Considering that any tracking techniques may produce false targets, to track the true target (in both DVP and COM) an operator stopped the automatic tracking procedure and performed a manual intervention to correct the software cursor position when the distance between the estimated center of the marker and the reference one (i.e., determined using the manual tracking) was higher than 4 pixels. The 4-pixel threshold was used because it almost corresponded to the radius of the markers (about 2 cm) for all the examined markers. It must be pointed out that a manual intervention was needed in those frames in which the software cursor went outside the marker’s edge because, in the following frames, the software would have not found the marker. This procedure was proposed to have a unique objective criterion of comparison. Given the multidirectional movements of the limbs and trunk during the performed activities, some markers were not visible in some frames. In such cases, the markers (n = 599, about 20%) were excluded from the calculation, i.e. the manual and automatic tracking were performed only on visible markers (n = 2940). One video (22 frames, 66 marker’s positions) was manually tracked five times by the same operator with the COM software. To assess the intra-operator reliability, the intra-class correlation (ICC, calculated through one-way analysis of variance), and the standard error of measurement (SEM) were used.

Assuming that a better automatic tracking procedure would require less human intervention, the percentage of manual interventions required for a given number of analyzed marker’s positions was used as the criterion to evaluate the effectiveness of the automatic tracking algorithm. The proportions of manual interventions observed in DVP and COM were compared using odds ratios (OR). The comparison was carried out separately for the different exercise modes and types of marker. Odds ratios were estimated from data of 2x2 contingency tables (software x manual intervention), as (n₁₁â‹…n₀₀) / (n₁₀â‹…n₀₁), where n₁₁, n₀₀, n₁₀ and n₀₁ are respectively: the number of marker’s positions not requiring manual intervention in COM, the number of marker’s positions requiring manual intervention in DVP, the number of marker’s positions requiring manual intervention in COM, and the number of marker’s positions not requiring manual intervention in DVP. Asymptotic interval confidences for odds ratios were derived from the standard error of the logarithm of the odds ratio, calculated as indicated by Agresti, 2002. All the statistical analyses were carried out using the R statistical software for windows (version 2.15.3).

The intra-operator reliability for the manual tracking procedure was high (ICC = 0.99, SEM = 0.449 pixels). The right knee flexion angle values obtained through the examined procedures for one subject in the “elliptical machine” exercise are shown, as an example, in Figure 3. In this case, there was no manual intervention in the great majority of frames for DVP, while manual interventions were frequent for COM. The angular values assessed with the two methods and with the reference manual tracking were very similar throughout the entire period considered (Figure 3).

For almost all the analyzed exercise modes, an OR significantly lower than 1 was observed (p < 0.05), meaning that the proportion of manual interventions was lower in DVP than in COM. Towed swimming represented the only exception, in which both software performed the automatic tracking practically without requiring manual interventions. To quantify the duration of both manual and automatic tracking processes, one video of towed swimming that did not require any manual intervention was taken as an example: the time required to perform the automatic tracking of 32 frames (96 markers) was 4min16s, that is, less than one half the time employed to manually track the same frames (10min40s).

For each examined exercise, the difference between the percentages of manual interventions required by both the DVP and the COM software ranged from 5.6% to 29.3% (Table 1). Considering the whole set of exercises, the proportion of manual interventions showed a difference of 10.4% between DVP (7.4%) and COM (17.8%). Similar results were found when the type of marker was analyzed. For full black circles, DVP required 9.9% less manual interventions than COM, while for the full white circles no manual intervention was required in both software (Table 2).

The aim of this study was to assess the effectiveness of a software developed for automatic tracking of features in underwater conditions. The assessment was based on the degree of automation of the software, calculated as the percentage of required manual interventions throughout the tracking process. The percentage of manual interventions for the examined videos ranged from 0.4% to 17% in the developed software, while it reached higher values (up to 46.3%) in the commercial software regarded as the reference (Table 1). Odds ratios computed by the number of frames automatically tracked and those that required a manual intervention in the two software were lower than 1 (p < 0.05), demonstrating that the automatic tracking of DVP had, for the whole sets of videos with exception of “towed swimming”, a higher degree of automation than COM. Furthermore, using different types of markers, a similar difference was observed (Table 2). Hence, the results showed an overall better automatic tracking effectiveness of DVP when compared to COM.

The different effectiveness observed for DVP and COM cannot be explained by the type of video recordings or by the operator’s markers identification criterion, because they were the same in both cases. Therefore, a likely reason for the difference observed between the two software lies into the algorithm used to track the markers. Indeed, analysing underwater movements presents a series of challenges due to intrinsic difficulties in obtaining images with overall good visibility. The DVP algorithm, derived from the KLT feature tracker (Lucas and Kanade, 1981, Tomasi and Kanade, 1991), exploits the overall flexibility provided by optical flow methods. This finds agreement with Shirageri et al., 2013 and Shukla and Patel, 2013. Reasonably, this characteristic allows the algorithm to overcome, at least partially, the difficulties of markers tracking in the underwater environment, resulting in a good degree of automation. Unfortunately, the automatic tracking method used by COM is not available for verification; therefore it was not possible to specifically individuate which characteristics of the two algorithms were more decisive to their different automatic tracking effectiveness.

The results relative to the single exercise modes can provide important elements for better understanding the automatic tracking effectiveness of DVP. For those videos with a high contrast between the markers and the background (for example, towed swimming) both software showed a very high degree of automation, with no manual intervention required. In the elliptical machine, on the contrary, a relatively high percentage of manual interventions were necessary for both software. This was due to the presence of background elements not clearly contrasting the marker (e.g., the foot support of the elliptical machine) that, being close to one or more markers in some phases of the movement, may have hindered the automatic tracking algorithms. However, a difference of 86 manual interventions (29.3%) was observed between COM and DVP, demonstrating that the tracking process of DVP tends to be more efficient even under disadvantageous conditions. This observation was confirmed by the analysis of software effectiveness when using different types of markers. No intervention was required by any software in the favourable condition represented by white circular markers on the black swimsuit. On the other hand, DVP required about 10% less interventions when considering full black circles as markers in all the videos together (Table 2). Thus, arranging an appropriate background contrast seems to be a key element for an effective automatic tracking. If this is not possible for some reason, using DVP may be a valid solution to reduce, at least partially, the processing time due to manual interventions for correcting the position of the cursor. The greatest advantage of the automatic tracking system is the elaboration time shortening, as demonstrated above, as automatic tracking of both COM and DVP was about 2.5 times faster than manual tracking. Furthermore, we extrapolate that the automatic tracking procedure of DVP is significantly less time consuming than that of COM when taking the whole video set into account due to the highest degree of automation of DVP. We can hypothesize that this is true also when the analysis does not concern an aquatic environment, and similar conclusions could be extended to the automatic tracking of movements performed outside the water.

The front crawl swimming (Figure 2C) and flutter kick (Figure 2G) exercise modes present the most undesirable characteristics of underwater recordings, with unavoidable bubbles and turbulences that negatively affect the visibility of the markers (Ceccon et al., 2013). Furthermore, the brightness of video recordings is unlikely to be constant in the presence of bubbles and turbulence, and this may cause more difficulty in the automatic tracking of markers. Actually, the analysis of these two exercises required a high number of manual interventions in both DVP and COM, thus showing a general limit of automatic tracking under conditions specifically related to a water environment. Once again, however, DVP required a lower number of manual interventions, demonstrating a higher degree of automation (OR = 0.67 and 0.34 for flutter kick swimming and front crawl swimming, respectively).

A further aspect related to the front crawl swimming exercise is that the movement was recorded from a multi-planar view (using six cameras) to perform the 3D analysis. In this case, due to the complexity of the movement and to the difficulty in identifying a joint’s or an anatomical landmark’s center, the automatic tracking process might require a specific manual supervision. A practical compromise between using more features, that means more accurate 3D motion analysis reconstruction, and a reasonably short elaboration time should be pursued. In this respect, results found in a previous study performed in dry condition with a stereo-photogrammetric system (Cappozzo et al., 1995) can be considered valid for swimming analysis (Ceccon et al., 2013).

The features tracking algorithm presented can be used in research involving underwater motion analysis, due to its good degree of automation and overall effectiveness in tracking different types of passive non- reflective markers when analysing underwater exercises.