Dynamic View Synthesis from Camera in Spinning Ball

Kodai Horita, Kris Kitani, Hideki Sasaki, Hideki KoikeThe Journal of The Institute of Image Information and Television Engineers 2014 · DOI

Abstract

We placed a small wireless camera inside an American football to capture the ball's view during flight to augment the spectator's experience when watching American football. To this end, we propose a robust video synthesis algorithm that leverages the unique constraints of fast spinning cameras to obtain a stable bird's-eye-view video clip. Our algorithm uses coarse-to-fine image homography computation to progressively register images. We then optimized the energy function defined over pixel-wise color similarity and distance to image borders to find optimal image seams to create panoramic composite images. Our results show that we can generate realistic videos from a camera spinning at speeds of up to 600 RPM.

Introduction

In sports broadcasting, capturing the "player's view" or "ball's view" is a primary goal for increasing viewer immersion. While cameras on helmets or surfboards are now common, putting a camera inside a ball presents a massive technical hurdle: high-speed rotation. An American football thrown in a spiral rotates at approximately 600 RPM (10 revolutions per second), turning raw footage into a dizzying blur. Furthermore, most small CMOS cameras use a "rolling shutter," which captures image lines sequentially. At these speeds, the rolling shutter causes extreme geometric distortion. This work introduces a framework to "de-spin" this footage, stabilize it, and expand the field of view (FOV) to create a usable "Ballcam" perspective.

Methodology

The proposed system utilizes a multi-stage pipeline to process high-speed video captured from a custom-built American football modified with an internal wireless camera.

Frame Selection and Rotation Phase Estimation

To create a stable downward view, the system must identify which frames (or parts of frames) are facing the ground. Since the sky is typically much brighter than the turf in an outdoor stadium, the algorithm uses the mean pixel intensity of each frame to estimate the ball's rotation phase and extract "ground-facing" frames.

Overview of the frame selection process, isolating ground-facing frames from the spinning ball's video stream.

Geometric Correction and Stitching

The pipeline addresses rolling shutter and lens distortion by incorporating the camera's rotation speed into the projection matrix. This allows for the simultaneous correction of "leaning" effects and wide-angle lens distortion. To expand the FOV, adjacent frames are stitched using a two-step coarse-to-fine homography approach. A linear translation model filters out outliers via RANSAC, followed by precise homography.

A key innovation is the use of a Graph-cut algorithm with a "Distance Cost" for seam finding. This forces the "cut" to stay near image edges, ensuring the central, most stable part of the primary frame is preserved while the edges are supplemented by adjacent frames. Finally, virtual intermediate frames are generated via motion interpolation to ensure fluid video output.

Results and Discussion

The system successfully produced stable video from balls spinning at up to 600 RPM. By stitching multiple frames (referred to as "triplets" or "sixlets"), the system significantly increased the field of view, providing a more immersive experience than a single narrow-angle frame. Qualitative results demonstrate that the geometric corrections effectively straightened the "leaning" lines of the football field caused by the rolling shutter, resulting in a perspective similar to a stabilized drone.

A key insight of this work is that the rolling shutter effect can be treated as a known geometric transformation based on the ball's RPM rather than mere noise. Additionally, using average frame brightness serves as a computationally efficient proxy for rotation phase, removing the need for high-speed external sensors.

Conclusion

This research demonstrates a robust framework for generating stable, bird's-eye-view footage from a fast-spinning projectile. While the current algorithm assumes a stable rotation axis, future work will focus on incorporating physical sensors (accelerometers/gyroscopes) to handle more complex ball movements and potentially integrating multiple cameras for a full 360-degree view.

Summary updated

Abstract

We placed a small wireless camera inside an American football to capture the ball's view during flight to augment the spectator's experience when watching American football. To this end, we propose a robust video synthesis algorithm that leverages the unique constraints of fast spinning cameras to obtain a stable bird's-eye-view video clip. Our algorithm uses coarse-to-fine image homography computation to progressively register images. We then optimized the energy function defined over pixel-wise color similarity and distance to image borders to find optimal image seams to create panoramic composite images. Our results show that we can generate realistic videos from a camera spinning at speeds of up to 600 RPM.
BibTeX
@article{horita2014dynamic,
  author = {Kodai Horita and Kris Kitani and Hideki Sasaki and Hideki Koike},
  title = {{Dynamic View Synthesis from Camera in Spinning Ball}},
  journal = {The Journal of The Institute of Image Information and Television Engineers},
  year = {2014},
  doi = {10.3169/itej.68.j495},
}
Figure