Most football (soccer) data in the world is an event log: a list of who touched the ball, where, and when. Each player has the ball for under two minutes of a 90-minute match (A public data set of spatio-temporal match events in soccer competitions), so these logs say almost nothing about the other 88 minutes. During that time defenders close passing lanes, teams hold their shape, and attackers make decoy runs. Full-pitch tracking data records every player continuously and would fill this gap, but it is mostly proprietary. The open tracking that exists covers only a handful of matches. The practical question is therefore a small-sample one: can a model learn to fill in the positions of unseen players from one or two tracked matches, and do it accurately enough to matter for analysis?
We propose to stop predicting each missing player on his own and instead predict the whole team as one deformable shape. A team's eleven positions move together in a few dominant patterns, such as shifting toward the ball, compressing, and stretching in transition. A low-dimensional "shape model" of those patterns has few parameters, so it can be estimated from very little tracking data. Unseen players are then recovered by fitting this shape to whatever is observed: the player on the ball at each event, and the visible players in broadcast freeze-frames. The fit is then refined under simple physical rules, namely bounded running speed and no two players occupying the same spot. This brings an idea that works in sample-starved medical imaging, a rough learned shape followed by physics-constrained refinement, into multi-agent reconstruction. In this setting nobody has yet tried it.
Core Claim
The closest prior work is Continuous football player tracking from discrete broadcast data. It trains a per-player autoregressive model on a single open Metrica match, where each player's motion is predicted from his own past with ball position as an outside input. Predictions are then matched to identities with the Hungarian algorithm and smoothed using estimated velocities. It reports a mean error of 7.33 m for off-camera players and 6.97 m near recorded events. The authors identify two failure modes: identity swaps in crowded set pieces, and error spikes after long stoppages. Both come from treating players independently and relying on recent temporal history.
Our claim is that a team-level shape prior reduces off-camera error and removes most set-piece swaps under the same one-match training budget. The prior constrains each player's position through his teammates' positions rather than through his own history alone, so it does not depend on that history after a restart. The approach borrows its design from A Left Ventricular Segmentation Method on 3D Echocardiography Using Deep Learning and Snake. On 45 ultrasound volumes, that work gets valid heart contours by combining a learned rough shape with an energy-minimizing deformable contour, sometimes called a "snake." The same paper also supplies a constraint: snakes fail when their starting point drifts. For us this means the shape must be re-anchored at every event rather than propagated freely through long stoppages.
Method
For team $k$ at time $t$, let $X_t \in \mathbb{R}^{22}$ stack the eleven players' pitch coordinates. We model these coordinates and estimate them as follows:
$$ X_t = \mu(b_t, h_t) + P\, z_t, \qquad \hat{X}_{1:T} = \arg\min \sum_t \Big[\, \|O_t (X_t - y_t)\|^2 + \lambda_z \|z_t\|^2 + \lambda_v \sum_i \phi(\|\dot{x}_{i,t}\|) + \lambda_r \sum_{i\neq j} \psi(\|x_{i,t}-x_{j,t}\|) \Big]. $$
The symbols are:
- $\mu(b_t, h_t)$ is the mean team shape given the ball location $b_t$ and the possession phase $h_t$. The phase is coarse: in possession, out of possession, or set piece.
- $P$ holds a few principal deformation directions, estimated from the training tracking match.
- $z_t$ is the low-dimensional shape code.
- $y_t$ are the observed positions, and $O_t$ selects which players are observed at time $t$.
- $\phi$ penalizes speeds above human sprint limits.
- $\psi$ penalizes players closer than about a metre to each other.
Identity assignment between observed freeze-frame players and slots in the shape is solved jointly inside the fit, again with the Hungarian algorithm. Because every slot is tied to its teammates through $P$, two players cannot simply trade places without distorting the team shape. We expect this to be the mechanism that suppresses set-piece swaps.
The second stage addresses data scarcity directly. StatsBomb 360 provides freeze-frames of visible players across more than 200 games, with team labels but without identities. These partial snapshots can be added to the estimate of $\mu$ and $P$ through the same assignment step. The risk is that other teams play different shapes, which would cause negative transfer. Following the instance-reweighting idea of Boosting Transfer Learning with Survival Data from Heterogeneous Domains, we weight each source frame by how well the current target shape model explains it. That work up-weights source patients who resemble the small target cohort and reaches a C-index of 0.677, against 0.500 for a target-only Cox model. We reuse only the weighting principle, not the boosted survival trees.
Research Program
We will build the shape model, the constrained fitting procedure, and the reweighted 360 extension. We will then test them in three ways. First, positional accuracy and identity swaps. Second, accuracy as a function of how much tracking data is used for training, down to fractions of a single match. Third, whether the reconstructions preserve the tactical quantities analysts would actually compute from them.
The third test is what links reconstruction back to the event-data problem. Event-only valuation such as DAxT in What Happened Next? Using Deep Learning to Value Defensive Actions in Football Event-Data cannot see off-ball defenders. It predicts the threat a disrupted attack would have produced from the two preceding events, and needs 760 matches to do so. Event-based expected-goals models likewise cannot see pressure on the shooter (Explaining Soccer Match Outcomes with Goal Scoring Opportunities Predictive Analytics, Explainable expected goal models for performance analysis in football analytics). Reconstruction only helps these models if its errors do not corrupt features like "defenders between ball and goal" or "nearest defender to the pass receiver."
We will not claim to fix valuation itself here. We test only the narrower and checkable claim that reconstructed context agrees with true context.
Baselines
The naive reference is the static or last-seen-position assumption: each unseen player stays where he was last observed. Continuous football player tracking from discrete broadcast data reports improving on such baselines by over 70%, but no reported absolute error for the static baseline was found on Lacuna. We will therefore re-run it ourselves on the identical observation protocol.
The strongest existing method is that paper's ARMAX plus Hungarian plus velocity-interpolation pipeline, with $\alpha \approx 0.6$. Its reported figure is a mean error of 7.33 m for off-camera players on Metrica Match 1 after training on Match 2. We will reproduce it with the same train/test split and the same rule for which players count as observed. We will tune both methods with an equal hyperparameter search budget on a held-out segment of the training match. Beating it means a lower off-camera mean error on Match 1 that exceeds the spread across leave-one-match-out folds over the open Metrica matches. Our target is a reduction of at least 20%, together with fewer identity swaps in corner and free-kick windows. A gain that appears only near events would not count, because event re-anchoring already explains that regime.
A third baseline is an unconstrained sequence model, a small transformer over all 22 players. It is trained on the same tracking data with the same tuning budget, plus standard augmentation by mirroring the pitch. It stands in for the default deep-learning answer.
Three ablations each remove one ingredient. Removing the shape term ($\lambda_z \to 0$, free per-player fit) tests whether the team prior is what matters. Removing the speed and spacing terms tests the physics refinement. Dropping the StatsBomb 360 freeze-frames, or including them without reweighting, tests transfer and negative transfer.
Why not just…?
Why not just use an average formation template? This is the most serious threat. A lookup of each player's mean position given ball zone and phase is our model with $P$ removed and no refinement, and it might capture most of the gain. We include it as an explicit baseline. If the template alone matches the full model within fold variance, the conclusion will be that deformation adds nothing, and the contribution reduces to "team-conditioned templates beat per-player dynamics." That is still a useful, simpler finding, and we would report it as such. The template has one specific weakness: it cannot reorder players dynamically. We therefore expect it to lose mainly in transitions and set pieces, and we report errors split by phase to test this.
Why not just train a bigger unconstrained network on the tracking data? With one or two matches, there are a few thousand strongly correlated frames from the same two teams. The small-sample evidence in the file is consistent that flat models underperform structured ones as data shrinks. MiME, which builds the clinical hierarchy into its architecture, beats flat baselines by 15% relative PR-AUC on the smallest EHR subsets (MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare). Target-only training collapses to chance in the TSB study. These are other domains, so we do not assume the result transfers. Instead we run learning curves from 10% to 100% of one match. These curves decide whether the transformer catches up, and at what data size.
Why not just track players from broadcast video? Off-camera players are not in the image, so a vision tracker cannot see them. Even on-screen, the full video-to-minimap pipeline reaches only 22.26% GS-HOTA, with about 56% jersey-number accuracy (SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap). Off-the-shelf ByteTrack scores 47.2 HOTA (SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos). Video detections are complementary as extra observed positions $y_t$; they are not a substitute for a prior over the unseen players.
Why not just augment the tracking data with diffusion-generated trajectories? DiffMDD shows diffusion augmentation helping small EEG cohorts (DiffMDD: A Diffusion-Based Deep Learning Framework for MDD Diagnosis Using EEG). A generator trained on one match, however, can only reproduce that match's two teams. Recombination-style augmentation also degraded the raw time-series model EEGNet in Assessing the Potential of Data Augmentation in EEG Functional Connectivity for Early Detection of Alzheimer's Disease. That warns that synthetic sequences can break temporal physics. Our approach gets diversity from real partial observations in 360 freeze-frames instead. We leave generative augmentation of the transformer baseline as an optional check rather than a core comparison.
Proposed Experiments
All accuracy experiments use the Metrica Sports open tracking and event data with leave-one-match-out evaluation. Observations are simulated following the broadcast protocol of Continuous football player tracking from discrete broadcast data. We report:
- mean and 90th-percentile positional error for off-camera players;
- error split by phase: open play, transition, set piece, and the first 10 seconds after a stoppage;
- identity-swap counts;
- learning curves over the amount of training tracking data.
The transfer experiment adds StatsBomb 360 freeze-frames to the shape estimate, with and without reweighting. We evaluate it on the same Metrica test matches, since 360 has no full ground truth.
The decisive downstream check compares tactical features computed from reconstructed and from true tracking on every Metrica pass and shot. The features are defenders goal-side of the ball, nearest-defender distance to the receiver, and whether the passing lane is blocked. We measure agreement with rank correlation and with accuracy on the lane-blocked label. This check is decisive because it ties reconstruction quality to the event-data blind spot that motivates the work. If the shape model lowers metre-level error but these features agree no better than they do under ARMAX, the gain is cosmetic. In that case the conclusion is that event-log valuation needs better observations, not better priors.
References
- A public data set of spatio-temporal match events in soccer competitions
- Continuous football player tracking from discrete broadcast data
- A Left Ventricular Segmentation Method on 3D Echocardiography Using Deep Learning and Snake
- Boosting Transfer Learning with Survival Data from Heterogeneous Domains
- What Happened Next? Using Deep Learning to Value Defensive Actions in Football Event-Data
- Explaining Soccer Match Outcomes with Goal Scoring Opportunities Predictive Analytics
- Explainable expected goal models for performance analysis in football analytics
- MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare
- SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap
- SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos
- DiffMDD: A Diffusion-Based Deep Learning Framework for MDD Diagnosis Using EEG
- Assessing the Potential of Data Augmentation in EEG Functional Connectivity for Early Detection of Alzheimer's Disease