Fly through View Video Generation of Soccer Scene

Naho Inamoto, Hideo SaitôIFIP 2003 · DOI · OpenAlex

Abstract

This paper presents a novel method for generating "fly-through" video sequences of soccer matches from arbitrary viewpoints using a limited number of stationary cameras. By segmenting the scene into dynamic (players), field, and background regions, the system avoids the computational complexity of full 3D reconstruction. The approach utilizes homography for the field, epipolar geometry for player morphing, and image mosaicking for the background. Experimental results demonstrate that synthesizing these layers independently and superimposing them creates realistic virtual camera paths with smooth transitions.

Introduction

Traditional sports broadcasting offers a fixed perspective, where viewers are restricted to camera angles chosen by a director. While advanced systems like "Eye Vision" have introduced multi-angle replays, they typically necessitate dozens of cameras and significant computational power to construct complete 3D models. In expansive and dynamic environments such as soccer stadiums, real-time 3D modeling of every player is computationally prohibitive and technically challenging due to calibration difficulties and complex movements. This research proposes an "arbitrary viewpoint" video generation technique that bypasses full 3D reconstruction by employing region-based view synthesis.

Methodology

The proposed system classifies the video scene into three distinct layers to simplify the synthesis process:

  1. Dynamic Regions (Players and Ball): Player silhouettes are extracted via background subtraction. Correspondence between different camera views is established using the ground's homography and epipolar geometry. Specifically, fundamental matrices (F-Matrix) define epipolar lines that guide pixel-wise matching, followed by image morphing to render players from intermediate angles.
  2. Field Region (Ground and Goals): The pitch is modeled as a series of planes. A homographic matrix ($H$) is calculated between real camera views to warp the field and goal structures into the desired virtual perspective.
  3. Background Region: Distant elements like stands are treated as an infinitely distant plane. A panoramic mosaic is constructed from the stationary cameras, from which the appropriate section for the virtual view is extracted.
Figure 1: Overview of the proposed region-based view synthesis pipeline.

The final arbitrary view is synthesized by superimposing these three layers. To ensure smooth camera movement, the system employs weighted linear interpolation between two real camera positions ($C_1$ and $C_2$), governed by an interpolation factor $\alpha$.

Figure 3: Geometry of virtual view synthesis between two real camera positions.

Experimental Setup

The method was evaluated using real-world footage from a soccer stadium. The setup consisted of four stationary video cameras positioned on one side of the field, targeting the penalty area. Input images were captured at a resolution of 720x480 pixels. For calibration, approximately 20 feature points were manually selected across the images to compute the necessary fundamental matrices and homographies.

Results and Discussion

The system successfully generated "fly-through" sequences where the virtual viewpoint moves seamlessly across the field. Qualitatively, the synthesized images appear natural, with no prominent artifacts at the boundaries of the superimposed regions. The use of weighted interpolation allows for fluid transitions between real camera viewpoints, effectively simulating a moving camera. A key advantage of this plane-based approach is its computational efficiency; since the field and background are static, they can be pre-processed, leaving only the dynamic player regions to be handled frame-by-frame.

Conclusion and Future Work

This research demonstrates a robust framework for arbitrary viewpoint synthesis in sports broadcasting without the need for dense camera arrays or complex 3D meshes. However, several areas for improvement remain. The current reliance on manual feature point selection for calibration is a bottleneck that the authors intend to automate. Additionally, handling occlusions in complex scenes where player silhouettes overlap remains a challenge for 2D morphing techniques. Future work will also focus on optimizing the pipeline for real-time performance to support live broadcast applications.

Summary updated

BibTeX
@inproceedings{inamoto2003fly,
  author = {Naho Inamoto and Hideo Saitô},
  title = {{Fly through View Video Generation of Soccer Scene}},
  booktitle = {IFIP},
  year = {2003},
  doi = {10.1007/978-0-387-35660-0\_13},
}
Figure