GolfDB: A Video Database for Golf Swing Sequencing

William J. McNally, Kanav Vats, Tyler Pinto, Chris Dulhanty, John McPhee, Alexander Wong

Abstract

The golf swing is a complex movement requiring considerable full-body coordination to execute proficiently. As such, it is the subject of frequent scrutiny and extensive biomechanical analyses. In this paper, we introduce the notion of golf swing sequencing for detecting key events in the golf swing and facilitating golf swing analysis. To enable consistent evaluation of golf swing sequencing performance, we also introduce the benchmark database GolfDB, consisting of 1400 high-quality golf swing videos, each labeled with event frames, bounding box, player name and sex, club type, and view type. Furthermore, to act as a reference baseline for evaluating golf swing sequencing performance on GolfDB, we propose a lightweight deep neural network called SwingNet, which possesses a hybrid deep convolutional and recurrent neural network architecture. SwingNet correctly detects eight golf swing events at an average rate of 76.1%, and six out of eight events at a rate of 91.8%. In line with the proposed baseline SwingNet, we advocate the use of computationally efficient models in future research to promote in-the-field analysis via deployment on readily-available mobile devices.

Introduction

Analyzing a golf swing is a fundamental aspect of the sport, yet it remains traditionally difficult to automate. Professional biomechanical analysis often requires expensive motion-capture setups, while manual video analysis is a tedious process requiring instructors to "scrub" through high-speed footage frame-by-frame. From a computer vision perspective, this is a "temporal event spotting" problem. While general action recognition has progressed rapidly, pinpointing the exact frame of a sub-action (sequencing) is significantly more challenging. Existing datasets have historically been too small, lacked high-definition quality, or failed to provide the specific temporal labels required for nuanced sports analysis.

The GolfDB Dataset

To address these limitations, we introduce GolfDB, a benchmark database designed for precision sequencing. The dataset consists of 1,400 high-quality (720p) videos sourced from YouTube, encompassing over 390,000 manually labeled frames. Each swing is annotated with eight key events: Address, Toe-up, Mid-backswing, Top, Mid-downswing, Impact, Mid-follow-through, and Finish.

GolfDB dataset composition by gender, club, view, and playback speed.

GolfDB is notable for its diversity, featuring 248 different professional golfers with varied swing styles. The metadata includes bounding boxes, player names, sex, club types (ranging from drivers to wedges), and camera views (face-on vs. down-the-line), providing a robust challenge for machine learning models.

Methods: The SwingNet Architecture

We propose SwingNet, a hybrid deep learning architecture designed for both accuracy and mobile efficiency. The model addresses the sequencing problem through two primary components:

  1. Spatial Feature Extraction: The system utilizes MobileNetV2 as a lightweight Convolutional Neural Network (CNN) backbone. It processes individual frames to extract visual features efficiently.
  2. Temporal Context: Identifying specific swing events, such as the "Top" of a backswing, requires understanding the frames immediately preceding and following the event. SwingNet passes CNN features into a Bidirectional LSTM (Long Short-Term Memory) network, allowing the model to evaluate the sequence in both forward and backward time.
The SwingNet architecture utilizes MobileNetV2 for frame-level feature extraction followed by a Bidirectional LSTM to capture temporal dependencies.

To manage the massive class imbalance—where critical events like "Impact" occur in only a single frame—we employed a weighted cross-entropy loss function during training. Furthermore, we found that using a smaller input resolution (160x160) allowed for larger batches and longer sequences, which yielded superior results compared to high-resolution images with shorter temporal context.

Key Results and Discussion

SwingNet achieved an average Percentage of Correct Events (PCE) of 76.1% across all eight events. When excluding the more subjective "Address" and "Finish" events—which even human experts find difficult to pinpoint—the accuracy rose to 91.8%. The "Impact" event was the most accurately detected, likely due to its high visual distinctiveness.

Efficiency was a core design goal. The "SwingNet-160" variant processes a single frame in approximately 10.6ms on a desktop CPU. This performance, paired with the lightweight MobileNetV2 backbone, makes the model an ideal candidate for real-time deployment on modern smartphones.

Conclusion and Future Work

While GolfDB and SwingNet provide a strong baseline for golf swing sequencing, several avenues for future research remain. Current analysis focuses on "trimmed" videos containing a single swing; future iterations must address "untrimmed" footage containing practice swings and other non-swing movements. Additionally, incorporating explicit "event order" logic (e.g., ensuring "Impact" follows "Top") could further refine accuracy. Expanding the dataset with pose or keypoint annotations would also enable deeper biomechanical insights, such as measuring joint angles or club head speed directly from mobile video.

Summary updated

Abstract

The golf swing is a complex movement requiring considerable full-body coordination to execute proficiently. As such, it is the subject of frequent scrutiny and extensive biomechanical analyses. In this paper, we introduce the notion of golf swing sequencing for detecting key events in the golf swing and facilitating golf swing analysis. To enable consistent evaluation of golf swing sequencing performance, we also introduce the benchmark database GolfDB, consisting of 1400 high-quality golf swing videos, each labeled with event frames, bounding box, player name and sex, club type, and view type. Furthermore, to act as a reference baseline for evaluating golf swing sequencing performance on GolfDB, we propose a lightweight deep neural network called SwingNet, which possesses a hybrid deep convolutional and recurrent neural network architecture. SwingNet correctly detects eight golf swing events at an average rate of 76.1%, and six out of eight events at a rate of 91.8%. In line with the proposed baseline SwingNet, we advocate the use of computationally efficient models in future research to promote in-the-field analysis via deployment on readily-available mobile devices.
BibTeX
@inproceedings{mcnally2019golfdb,
  author = {William J. McNally and Kanav Vats and Tyler Pinto and Chris Dulhanty and John McPhee and Alexander Wong},
  title = {{GolfDB: A Video Database for Golf Swing Sequencing}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=OOicEUDkwP},
}
Figure