A virtual coaching environment for improving golf swing technique

A Virtual Coaching Environment for Improving Golf Swing Technique Philip Kelly, Aoife Healy, Kieran Moran and Noel E. O’Connor∗ CLARITY: Centre for Sensor Web Technologies, Dublin City University, Ireland kellyp@eeng.dcu.ie ABSTRACT Keywords As a proficient golf swing is a key element of success in golf, many golfers make significant effort improving their stroke mechanics. In order to help enhance golfing performance, it is important to identify the performance determining factors within the full golf swing. In addition, explicit instructions on specific features in stroke technique requiring alterations must be imparted to the player in an unambiguous and intuitive manner. However, these two objectives are difficult to achieve due to the subjective nature of traditional coaching techniques and the predominantly implicit knowledge players have of their movements. In this work, we have developed a set of visualisation and analysis tools for use in a virtual golf coaching environment. In this virtual coaching studio, the analysis tools allow for specific areas require improvement in a player’s 3D stroke dynamics to be isolated. An interactive 3D virtual coaching environment then allows detailed and unambiguous coaching information to be visually imparted back to the player via the use of two virtual human avatars; the first mimics the movements performed by the player; the second takes the role of a virtual coach, performing ideal stroke movement dynamics. The potential of the coaching tool is highlighted in its use by sports science researchers in the evaluation of competing approaches for calculating the X-Factor, a significant performance determining factor for hitting distance in a golf swing. Content Analysis, 3D Graphics, Data Visualisation, Golf, Sports Performance Categories and Subject Descriptors H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems—Animations, Artificial, augmented, and virtual realities; J.m [Computer Applications]: Miscellaneous General Terms Design, Experimentation, Performance ∗ This work is supported by Science Foundation Ireland under grant 07/CE/I1147. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SMVC’10, October 29, 2010, Firenze, Italy. Copyright 2010 ACM 978-1-4503-0175-6/10/10 ...$10.00. 1. INTRODUCTION Golf is a precision sport played by 10-20% of the adult population in most countries [11]. An accomplished golf swing is a key component of success in golf. As such, many amateur players spend a considerable amount of time and effort perfecting their golf stroke mechanics – hoping to create more accuracy, consistency and distance in their swing. However, in order to swing a golf club effectively involves precision complex motor activity condensed into a short space of time. In addition, a tremendous amount of golf specific strength, flexibility, co-ordination, balance and stability is required. In order improve a golf swing, all the faults in a player’s stroke dynamics must be both identified and eliminated – starting with larger faults in technique, and finally fine tuning to remove smaller issues. However, these two complementary objectives of identification and elimination are difficult to achieve in practice. Take for example, the studies of Vic Braden, a tennis coach to some of the greatest players in the history of the game, into what makes a professional tennis player’s forehand stroke better than that of amateurs. In [10] Braden is quoted as saying that “Almost every pro in the world says that he uses his wrist to roll the racket over the ball when he hits a forehand”. This is backed by players such as Andre Agassi who stated “I take the ball on the rise and play with a lot of wrist” [1]. On visual inspection this perception appeared to be correct. However, after extensive studies with high-speed camera equipment Braden found that most professionals, including Agassi, almost never moved their wrists until well after the ball was hit. Clearly something is making a professional tennis player’s forehand stroke better than an amateur’s; but it isn’t rolling the racket over the ball! This example illustrates how problems can arise when players, of any sport, are asked to describe how they performed athletically. They can be fundamentally mistaken about the identification of the performance determining factors. In a similar vein, players of a lower level can misinterpret the areas requiring improvement in their technique. One possible explanation is that athletes have mainly implicit knowledge of their movements [17]. Implicit knowledge is defined in [16] as that which is revealed in task performance without any corresponding phenomenal awareness. They are the skills that people have that cannot be put into words. Explicit and implicit knowledge are similar in meaning to conscious and unconscious knowledge. Explicit knowledge refers to that expressed as conscious experience and that people are aware that they possess. Implicit knowledge, by contrast, refers to knowledge that is not verbally reportable [17]. In this work, we have developed visualisation and analysis tools for the identification and elimination of faults in a golf player’s stroke mechanics via the use of an immersive virtual 3D coaching environment. In order to achieve these goals the implicit movements of a test subject’s golf stroke must be analysed and, from this information, explicit instructions on how the player should change their stroke mechanics must effectively imparted to the user. The proposed approach extracts these explicit instructions by simultaneously comparing the implicit movements of the player in question, captured using a high speed motion capture system, to a number of time-aligned swings from players with higher determined skill levels. Using this approach, the performance determining factors within golf strokes can be identified from the higher skilled players – this knowledge can then be employed to isolate and report the major differences in the stroke dynamics of the lower level player. In order to eliminate these faults, explicit instructions on how to remove identified faults from their stroke must be imparted to the player in an unambiguously and intuitive manner. In this work, this is achieved by rendering the movements of the player, a coach and explicit 3D visual aids in a virtual 3D coaching studio. In this 3D environment, the actions preformed by the coach avatar are determined by the average movements of all the high-skilled players – as such, it illustrates the ideal swing that the player should try to mimic. A virtual human avatar also depicts the motion of the real-world player, thereby allowing a visual analysis of the differences between the two strokes to be made. Furthermore, explicit instructions can be unambiguously imparted to the user by overlaying 3D visual aids that focus the user’s attention to important technical corrections that the player must make to improve their swing. The paper is organised as follows: Section 2 provides a brief overview on previous work conducted in the area of the scientific analysis of golf swings. Section 3 gives an overview of the data collection undertaken in this work. Section 4 details how timing variations between differing players and strokes, which occur due to varying tempos of swing, are eliminated in the proposed system – these timing variations must be eliminated in order to make meaningful insights into how a player’s stroke differs from those of higher skilled golfers. Section 5 details how the implicit movements of a test subject’s stroke is analysed, how areas requiring improvement are isolated and how explicit instructions are provided to the player through the means of a virtual immersive environment. In section 6 a discussion on the adoption of the system’s golf swing analysis tools – the section of this work pertaining to the comparison of strokes from multiple high level players in order to isolate performance determining factors – as a means of gathering important information for data analysis in a separate study in the area of sports science is presented. Finally, conclusions and future work are outlined in section 7. 2. RELATED WORK The amount of rigorous scientific research that has been conducted into golf is surprisingly limited. A number of (a) 3D markers (b) Model fit to markers (c) 3D model Figure 1: Visualisation in virtual environment. books discussing the biomechanics of golf have been written by professionals and coaches, although these usually lack scientific foundation and are mainly based on personal experience and opinion [11]. The majority of the previous scientific research has tended to restrict its analysis to only a small number of biomechanical factors throughout a golf swing, for example [7] and [9] focus solely on one and two factors respectively. Other studies restrict their analysis to only three distinct events in a golf swing (TA, TB and BC – see Figure 3). Overall, the majority of previous research in this area has aimed at identifying performance determining factors in golf swings made with a driver golf club [15, 12], despite the fact that either an equal or even a higher proportion of shots for maximum distance in the game of golf are taken with iron clubs. In order to address this issue, in this work we focused our analysis on full golf swings aiming to hit the maximum distance using a 5 iron club. It should be noted however, that the proposed virtual environment can be applied to full golf swings using any club. 3. DATA COLLECTION AND SKILL LEVEL CLASSIFICATION In this work, 40 male right handed and injury free golfers, aged 33±15.43 years with an average handicap of 7.93±5.46, were recruited from local golf clubs. Forty one reflective spherical markers were placed on anatomical landmarks on the body in positions predescribed by sports science researchers, in addition 3 markers were placed on the golf club, see Figure 1(a). A 12 camera 250 Hz Vicon infra-red motion capture system [4] was used to record the 3D motion of the participant throughout the golf swing. The Vicon system is a semi-automated motion capture system that tracks the 3D position of infra-red reflective markers in 3D space with a high degree of accuracy (up to 1 mm in a 6 metre space). The testing session consisted of a prescribed warm up, recording of fifteen golf swings and a participant selected cool down period. The prescribed warm up consisted of five minutes walking on a treadmill (2.5 km/h) followed by 3 minutes of practice swings. The participants were instructed to “hit the ball as hard as possible towards the target-line, with the aim to maximise both distance and accuracy, as if in a competitive situation” into a net located three metres from the swing analyser using their own 5 iron golf club. Given an arbitrary player, in order to compare their golf stroke movement dynamics to those of higher-skilled players, each player in the capture dataset needs to be graded according by their relative golfing aptitudes. In an ideal scenario, each participant’s skill level would be graded by the distance the ball travelled and the accuracy of the stroke. However, within a laboratory setting it was not possible to measure these parameters. As such, ball speed was chosen (a) Unaligned LB (b) Unaligned ED (c) Unaligned MD (d) Unaligned BC (e) Aligned LB (f) Aligned ED (g) Aligned MD (h) Aligned BC Figure 2: Apprentice is depicted on the right in white. (a-d) Unaligned Swings; (e-h) Swing timings aligned. (TA) Takeaway (MB) MidBackswing (LB ) LateBackswing (TB) TopBackswing (ED) EarlyDownswing (MD) MidDownswing (BC ) (MF ) MidBall-Contact FollowThrough Figure 3: Eight key events in a golf swing. to discriminate between player skill levels at it is one of the strongest determinants of the distance the ball travels. In order to obtain this ball speed, each stroke the ball was hit from a tee on a Pro V swing analyser [2], which can be used to measure the launch characteristics of the ball (such as the impact point) and the club face properties (such as angle and speed) at the time of impact. Participants were sorted according to their average ball speed for their fifteen golf swings. The authors acknowledge that this method of grading participants’ golfing performance using ball velocity is not without limitations. While the ball velocity is a major factor in determining the distance the ball travels it does not take into account the accuracy of the shot. However results showed that if the players were split into two groups; (1) a high speed group (HSG) with a stroke speed above the average of all the players; and (2) a low speed group (LSG) with a stroke speed below the average; then it was shown that the HSG hit the ball significantly closer to the centre of the club face (-0.74 cm vs. -1.95 cm, where a negative value indicates the impact point is towards the heel of the club head) than the LSG, indicating that a higher level of accuracy also exists within the HSG. 4. STROKE ALIGNMENT In order to provide feedback to a player in the virtual 3D environment, the implicit movements between that given player and those of higher skill need to be examined. However, before the implicit movements between differing strokes can be compared, the timing variations between them must be eliminated. These timing differences can occur due to contrasting tempos of swing. For example, in the top row of Figure 2, in (a) the two players are relatively close in the phase of their swing – both player’s have just turned from an backswing to a downswing. By (b)-(d) however, the phase of their swings has rapidly diverged, for example in (c) the right player has already hit the ball whereas the left player is only at the mid-downswing position of their stroke. In each of these figures, a 3D model is fit to the 3-D location of the spherical markers within a given frame and then rendered in real-time using OpenGl [3], see Figures 1(b) and (c). The first stage in this work is to remove the timing variations caused by contrasting swing tempos, thereby making it possible to compare different swing techniques at arbitrary times throughout the swing. Let the golfer being examined for faults in their swing dynamics be known as the apprentice, and a player of higher skill level be designated as an expert. In order to time align the two players, the stroke timing of either the apprentice or expert should be warped so that the swing tempos of the two players coincide. In general, we believe that the apprentice golfer should remain unaltered so as when the apprentice’s stroke is visualised by a 3D avatar, the tempo of their stroke remains unchanged and, as such, remains familiar to the apprentice thereby increasing the possibility of the player being able to eliminate the discovered flaws. The alignment of swing tempos is achieved by in the proposed system by automatically time-aligning different strokes at eight functionally key events throughout the swing via a dynamic time warping approach. These eight key events were adapted from [5], as shown in Figure 3. The events are automatically detected in the proposed software using the positing and velocity of the three reflective markers placed on the golf club – for example; Takeaway (TA) occurs when the player initially starts their swing (when the velocity and direction of movement of the markers indicate the club is consistently being brought backwards to indicate the start of the backswing); MB, MD and MF all occur when the three club markers are equidistant from the groundplane; LB and ED occur when the three markers lie perpendicular to the groundplane; and finally TB occurs when the velocity of the markers indicate a change from a backswing to a forward swing (their velocity will drop to zero and then accelerate in a reverse direction). The first stage in the alignment stage determines the temporal location of the TA event within the apprentice and expert sequences and offsets the start temporal time of the expert sequence. As such, if the apprentice and expert sequences are played in parallel, the TA event will occur at the same point in time. Then in order to temporally align a full sequence, the apprentice stroke is segmented into 7 segments, m1..7 , where m1 refers to all frames between the first two events namely TA and MB, m2 refers to all frames between the second two events MB and LB, etc. Similarly, the expert stroke is segmented into 7 corresponding segments, s1..7 . Each independent expert segment, si , is linearly spedup or slowed-down in the temporal domain in order to make the duration of si in time to be equal to that of mi . The results of the dynamic time warping process can be seen in Figures 2(e)-(h). (a) Left Shoulder Y-Angle Comparison 5. FAULT IDENTIFICATION AND 3D VISUALISATION Once temporally aligned, comparison of implicit movements between swings can be made and clear performance determining factors can be uncovered and visualised. Using the aligned motions, the proposed system analysis a large number of variables throughout the full swing sequence – these variables include the angles and angular velocities of the shoulders, elbows, wrists, hips and knees. In addition, throughout the swing the X-Factor is evaluated. The XFactor describes the relative rotation of the torso with respect to the pelvis during the golf swing. McLean [13] first demonstrated that the greater the X-Factor at the top of the backswing, the higher a professional was ranked on driving distance. Subsequent research also supports the importance of the X-Factor [9, 14]. When presented with two players, an apprentice and an expert, this information is doubled for all points in time throughout the swing (one set of angles, velocities and XFactor for each player). All this data can quickly cause information overload to the user, making it more difficult, not easier, to identify the underlying signature of a good golf swing. For example, say at time t, there is a 1 degree difference in right shoulder rotation, a 5 degree difference in the left wrist, etc. Which difference is most significant with respect to performance? With just one expert, this is difficult to tell. However, given multiple experts, each one a player of a higher skill level than the apprentice, a pattern can emerge in the common factors that exist between all the experts but lacking in the apprentice. Returning to the previous example, if there is always a small difference (say ±0.5 degrees in standard deviation around the mean) in the right shoulder angle between all experts whereas there is a standard deviation of ±10 degrees for the left wrist, then the 1 degree difference in shoulder angle can be deemed to have the highest likelihood as a performance determining factor. The more experts in the comparison, the higher this likeli- (b) Right Knee Y-Angle Comparison Figure 4: Immersive Virtual Coaching Environment. hood becomes. Using this proposed methodology, whereby angles, velocities and the X-Factor are ranked in terms of a significance factor that is determined using the implicit motions from multiple experts, explicit instructions can be obtained on how the apprentice should alter their swing in order to gain the greatest improvement in technique. In this work, these explicit instructions are unambiguously and intuitively imparted to the user by means of two virtual avatars and a corresponding 2D plot of the feature in question. The first avatar positioned in a virtual coaching environment represents an accurate 3D graphical rendering of the apprentice player’s swing (see the avatar in blue in Figure 4). The second avatar (pink in Figure 4) illustrates the average position of all expert players used in the comparison, we label this second avatar as the coach. The coach provides a visualisation of the ideal swing that the player should try to mimic. In this virtual environment the user has complete control of the view angle, zoom level and positioning. In addition, the user controls the speed of playback for the golf swing – thereby allowing the sequence to be viewed in realtime, fast forward or slow motion (down to 250 times slower than real-time). Furthermore the user also has the ability to can jump the animation to key events, pause, fast-forward and rewind at will by clicking and dragging on a time line. Given the amount of visual information available in the rendered environment, it is essential to be able to key the user’s attention to important movement information in the apprentice’s stroke dynamics that need adjusting. This is facilitated by the use of additional 3D visual aids. For example, in Figure 4(a) the left shoulder Y-angle of the apprentice requires alteration, as such a red 3D arrow indicating the direction of change required has been overlaid onto the player’s avatar. Similarly, later in the stroke the extension of the right knee needs to be adjusted – see Figure 4(b). The 2D graphs in both figures can also be used to trace the respective angle of interest for the apprentice and the coach throughout the entire stroke – the current temporal position along the graph is indicated by the vertical red line. This feature allows the user to observe how and where they diverge from the stroke of the coach. 6. SPORTS SCIENCE EVALUATION STUDY In addition to the use of the proposed framework for imparting valuable feedback data on stroke dynamics to golfers, it has also been employed as a basis for obtaining important information for data analysis on by a sports science research team. Using data gathered by the infrastructure, a number of general rules were obtained to which players should adhere in order to increase their ball speed. These included; increasing shoulder flexion and elbow extension to create a greater arc for the club head to travel through, thus generating greater club head speed; creating a greater range of movement in the shoulders and hips leading to greater angular velocities at these joints and subsequently greater club head and ball speeds; extending the right hip from MD to MF to aid in the faster transfer of weight to the front foot; and flexing shoulders more during the backswing, thereby utilising a greater range of motion in the backswing leading to greater angular velocity in both shoulders at early downswing. Many of these rules are backed in general literature describing golf technique [8, 6], furthermore novel performance determining factors were also identified [11]. The proposed system was also employed by the sports science research team to conduct analysis on, and visualise, competing approaches for measuring the X-Factor in golf players. The X-Factor describes the relative rotation of the torso with respect to the pelvis during a golf swing and has been identified as a significant performance determining factor in a golf swing [13, 9, 14]. However, this research has used a simplified and potentially inaccurate means of calculating the X-Factor using the Projection Method (PM). The PM describes the X-Factor as the angle between the torso and pelvis axes when both are projected onto the global horizontal groundplane – see Figure 5 (1a)-(1e). When standing upright, the rotation about the longitudinal axis of the pelvis and the torso are in the global horizontal plane, which is the plane in which the X-Factor projection method is calculated. However, in golf a forward tilting posture of the pelvis and torso occurs, which results in the horizontal plane of the body segments no longer being parallel to the global horizontal plane. As the movement of the body during the golf swing does not solely occur in the global horizontal plane measuring the X-Factor, errors may be introduced when calculating the X-Factor using the PM approach. Using the proposed system as a data collection framework, a study was undertaken to assess this error introduced by the PM by comparing it to a more appropriate method, namely the X-Factor Segmental Method (SM), and to examine if the error introduced by the PM is consistent throughout the swing, and so could be post-processed to remove the error. The SM is calculated by obtaining the differential between the rotation of the torso and pelvis body segments about their own longitudinal axis – see Figure 5 (2a)-(2e). To the authors’ knowledge, no previous research has been undertaken that compares these methods of X-Factor calculation. The result of this study indicated that the PM significantly over-estimates the X-Factor angle in comparison to the SM approach. In addition, the differences were not consistent throughout the swing, with the largest absolute difference evident at the top of the backswing and reducing in size the closer the club is to the ball – see Figure 6. In conclusion, given that the difference appears dependent upon body posture, as evident by the significant differences at different events during the swing, it is not possible to apply a mathematical “correction factor” to rectify the error in the PM X-Factor angle. As such, it is recommended that while more difficult to measure, the SM is functionally more representative than the PM and therefore should be used in giving feedback to golfers. 7. CONCLUSIONS AND FUTURE WORK In this paper, we presented visualisation and analysis tools for the identification and elimination of faults in a golf player’s stroke mechanics. In order to achieve these goals the implicit movements of a test subject’s golf stroke are aligned and compared to multiple players of a higher skill level. From these comparisons explicit instructions on how the player should change their stroke mechanics are extracted and visualised in a 3D virtual coaching environment. The successful use of the proposed system by a sports science research team, for extracting performance determining factors, exhibit its potential effectiveness as a coaching tool. In this work all input data consisted of high speed 3D motion capture data. While this system is highly accurate, it is also both very expensive and requires expert users to operate it. In future work, we will investigate the use of alternative forms of data acquisition, for example the use of cheap, light accelerometers on various player limbs. Using this form of input, plus a calibration stage, the proposed system could be adapted for use by non-expert users without the need for constraining motion capture rigs. Finally, we would like to adapt the framework for a variety of other sports, such as tennis or cricket. This could be achieved by either adjusting the motion alignment algorithm to align alternative key events or by using a dynamic programming based approach to align each movement frame independently. 8. REFERENCES [1] [2] [3] [4] [5] Inside tennis, Oct/Nov 2001. http://www.golftek.com/pro5.html, 2010. http://www.opengl.org, 2010. http://www.vicon.com, 2010. K. Ball and R. Best. Different centre of pressure patterns within the golf stroke II: Group-based analysis. Sports Sciences, 25(7):771–779, 2007. [6] M. Broer. Efficiency of Human Movement. W.B. Saunders, 1973. [7] D. Budney and D. Bellow. On the swing mechanics of a matched set of golf clubs. Research Quarterly for Exercise & Sport, 53(3):185–192, 1982. (1a) (1b) (1c) (1d) (1e) (2a) (2b) (2c) (2d) (2e) Figure 5: Row 1: Projection Method ; (1a) Torso axis; (1b) Project axis onto groundplane; (1c) Projected Torso axis; (1d) Projected pelvis axis; (1e) X-Factor angle, θ, is that which lies between the two axes. Row 2: Segmental Method ; (2a) T-Pose; (2b) T-Pose Frontal and Longitudinal axes of both Pelvis and Torso; (2c) Align T-Pose and Stroke-Pose Longitudinal (blue) axes, angle α between two Frontal axes; (2d) Similarly angle β obtained for pelvis; (2e) X-Factor angle is α − β. Figure 6: Comparison of PM and SM X-Factor angles (graphed in red and blue respectively) for a single swing at the 8 key events. [8] J. Bunn. Scientific principles of coaching (2nd ed.). Prentice-Hall, 1972. [9] P. Cheetham, P. Martin, R. Mottram, and B. St. Laurent. The importance of stretching the “x factor” in the downswing of golf. Optimising Performance in Golf, pages 192–199, 2001. [10] M. Gladwell. Blink. New York: Little, Brown and Company, 2005. [11] A. Healy. Identification of the biomechanical performance determining factors of the 5 iron golf swing when hitting for maximum distance. Master’s thesis, School of Health and Human Performance, Dublin City University, Ireland, 2009. [12] P. McLaughlin and R. Best. Three-dimensional kinematic analysis of the golf swing. In Science and golf II: Proceedings of the 1994 World Scientific Congress of Golf, pages 84–90, 1994. [13] J. McLean. Widen the gap. Golf Magazine, pages 49–53, December 1992. [14] J. Myers, S. Lephart, and Y. Tsai. The role of upper torso and pelvis rotation in driving performance during the golf swing. Journal of Sports Sciences, 26(2):181–188, 2008. [15] R. Robinson. A study of the correlation between swing characteristics and club head velocity. In Science and golf II: Proceedings of the 1994 World Scientific Congress of Golf, pages 84–90, 1994. [16] D. Schacter. Implicit knowledge: New perspectives on unconscious processes. Proceedings of the National Academy of Sciences of the Unites States of America, 89:11113–11117, 1992. [17] M. Sylvester. Teaching the unknowable: Does analogy lead to implicit skill acquisition in a dart-throwing task? Master’s thesis, Queen’s University, Canada, 2007.