Mitigating Motion Blur for Robust 3D Baseball Player Pose Modeling for Pitch Analysis

Jerrin Bright, Yuhao Chen, John S. Zelek

Abstract

Accurate 3D pose estimation of baseball pitchers is critical for performance analysis and injury prevention in professional sports. However, standard broadcast video, typically recorded at 30 frames per second (fps), introduces significant motion blur and self-occlusion challenges that degrade the performance of traditional computer vision models. This paper presents a robust pipeline that utilizes a novel synthetic motion blur augmentation strategy and "in-the-wild" data integration to improve pose modeling. By teaching models to recognize and interpret motion blur rather than attempting to remove it, the proposed framework achieves a 54.2% reduction in 2D pose loss and a 36.2% reduction in 3D pose loss compared to baseline methods.

1. Introduction

In the era of Sabermetrics, the visual analysis of a pitcher’s mechanics has become a cornerstone of player development. While high-speed camera systems provide the fidelity required for precise biomechanical tracking, their high cost and logistical requirements limit their accessibility. Standard broadcast footage offers a massive, untapped repository of data, but its low temporal resolution (30fps) presents a major hurdle: the high-velocity movement of a pitcher’s arm results in significant motion blur. Furthermore, the standard "home plate view" often leads to self-occlusion, where the pitcher's body hides critical joints. This research addresses these limitations by developing a 3D pose estimation pipeline resilient to real-world broadcast artifacts.

2. Methodology

The proposed system transitions from raw 2D broadcast frames to a full 3D body mesh through a specialized multi-stage architecture.

2.1 Motion Blur Learning Module

Rather than employing computationally expensive deblurring pre-processing, the authors developed a strategy to make the model inherently robust to blur. Using a RAFT-based optical flow algorithm, the system identifies specific image patches characterized by high motion. A parameterized motion blur filter—controlled by rotation ($\omega$) and scale ($s_f$)—is then applied to these patches during training. This "patch-based" approach ensures the model learns to associate smeared pixels with joint locations in a way that mimics real-world physics.

2.2 In-the-Wild Data Integration

To enhance the model's ability to generalize across different environments and lighting conditions, the researchers curated high-resolution, slow-motion pitching videos from public internet sources. By generating "pseudo-ground truth" poses from these clear videos and then synthetically introducing motion blur, they created a diverse training set that bridges the gap between laboratory-quality data and low-quality broadcast footage.

2.3 The Pose Estimation Pipeline

The architecture consists of three primary components:

  1. 2D Estimation: A regressor-based model (PEFormer) predicts 2D keypoints. The researchers found that direct coordinate regression is significantly more effective than heatmap-based methods in blurry conditions, as heatmaps often struggle with the smeared signals of a moving limb.
  2. 3D Lifting: A vision transformer (MHFormer) is utilized to "lift" sequences of 2D poses into a 3D coordinate space, maintaining temporal consistency.
  3. Mesh Reconstruction: A Spectral Graph Convolutional Network (GCN), specifically Pose2Mesh, combines 2D and 3D data to estimate a comprehensive 3D body mesh.

3. Experimental Setup

The study utilized a primary baseball dataset encompassing over 1,000 games and 100,000 pitches, featuring 18 3D joint positions and synchronized pitch metrics. For the core experiments, a sub-sample of 150 pitch sequences (approximately 30,000 frames) was used, partitioned into training (105), validation (15), and testing (30) sets. This was supplemented by the aforementioned "in-the-wild" slow-motion data to provide a robust benchmark for the augmentation strategy.

4. Results and Discussion

The integration of the motion blur augmentation strategy yielded substantial performance gains across all metrics. On the test dataset, the framework achieved a 54.2% reduction in 2D pose loss and a 36.2% reduction in 3D pose loss. When applied to existing state-of-the-art (SOTA) pose estimators, the method provided an average performance boost of 29.2%.

4.1 Key Insights

  • Augmentation Efficiency: Focused augmentation (adding blur to training) proved more effective and computationally efficient than complex deblurring architectures.
  • Regression Robustness: In scenarios where joints are obscured by motion, regression-based estimators provide more stable predictions than probability-based heatmaps.
  • Optimal Parameterization: Ablation studies indicated that using two filters and a patch size of 30 provided the optimal balance for learning motion characteristics without overfitting to noise.

Qualitatively, the model demonstrated the ability to reconstruct 3D meshes even when the pitcher's arm was almost entirely obscured by blur—a scenario where previous models like ICON or OSX typically failed to maintain tracking.

5. Conclusion and Future Work

This research demonstrates that motion blur in broadcast video can be mitigated through intelligent synthetic augmentation rather than complex pre-processing. By focusing on patch-based blur and regression-based estimation, the pipeline provides a viable path for professional-grade biomechanical analysis using standard video. Future work will explore the incorporation of more detailed body shape information and the use of contextual scene cues to further improve accuracy during extreme occlusions. Additionally, the authors suggest that synthesizing multiview data from single-view broadcasts could further refine the 3D fidelity of the models.

Summary updated

BibTeX
@inproceedings{bright2023mitigating,
  author = {Jerrin Bright and Yuhao Chen and John S. Zelek},
  title = {{Mitigating Motion Blur for Robust 3D Baseball Player Pose Modeling for Pitch Analysis}},
  booktitle = {MMSports@MM},
  year = {2023},
  doi = {10.1145/3606038.3616163},
}
Figure