Synthetic data for sharing and exploration in high performance sport

John Warmenhoven, Franco M. Impellizzeri, Ian Shrier, Andrew D. Vigotsky, Lorenzo Lolli, Paolo Menaspà, Aaron J. Coutts, Maurizio Fanchini, Giles Hooker2024 · DOI

Abstract

Synthetic data represent alternative data sources generated using mathematical procedures to address specific issues in research and practice. In high-performance sport, synthetic data has emerging applications for overcoming privacy barriers and supporting Open Science. This study explores the applicability of sequential tree-based algorithms (via the synthpop package in R) using a professional football dataset. Through seven simulation examples, we demonstrate that while synthetic data can effectively mimic reference datasets, its utility depends heavily on the alignment between the synthesis model and the intended analysis. We propose a framework of three conceptual models—analysis, synthesis, and true generation—to guide researchers. Because synthetic data is not a direct replacement for real data, we emphasize the need for rigorous documentation regarding the generation process and its boundary conditions.

Introduction

In the era of "Big Data," high-performance sports organizations are increasingly pushed toward FAIR (Findable, Accessible, Interoperable, and Reusable) data practices. However, the field faces a unique paradox: while open data can accelerate sports science research, athlete data is highly sensitive. Athletes are easily re-identifiable due to their public profiles, and professional teams often view performance data as a proprietary commodity that provides a competitive edge.

Synthetic data—datasets generated by algorithms to mirror the statistical properties of a real dataset without containing actual individual records—offers a potential bridge. This paper evaluates the effectiveness of synthetic data in the context of longitudinal athlete monitoring, specifically focusing on training loads and injury patterns.

Methodology

Data Source

The study utilized a longitudinal dataset from Fanchini et al. (2019), tracking 34 professional football players over three consecutive seasons. Key variables included WeekID, PlayerID, Acute Load (weekly training load), Chronic Load (a 4-week rolling average), and a binary Injury status.

Synthesis Process

Researchers employed the synthpop package in R, utilizing Classification and Regression Trees (CART). To understand the nuances of data generation, the team tested seven distinct simulation conditions. These conditions varied the "visit sequence" (the order of variable generation) and the predictors used. Notably, the simulations ranged from a "Base" model (matching the variables used in the original study's analysis) to complex models incorporating time-lagged variables (e.g., training loads from 1–3 weeks prior) to capture temporal structures.

Evaluation Metrics

Utility was assessed through two lenses:

  1. Global Utility: Measured via propensity score Mean Squared Error (pMSE), determining if the synthetic data is statistically indistinguishable from the original distribution.
  2. Specific Utility: Measured by the Mean Absolute Error (MAE) of parameter estimates from a Generalized Estimating Equation (GEE) used to predict injury from training load.

The Three-Model Framework

A core contribution of this research is the conceptualization of three distinct models involved in the synthetic data lifecycle:

  1. The Analysis Model: The specific statistical model used to answer a research question (e.g., the GEE for injury prediction).
  2. The Synthesis Model: The mathematical procedure used to generate the synthetic data.
  3. The True Generation Process: The underlying biological or environmental reality that created the original data.

Biases and errors typically emerge when these three models are misaligned. If the synthesis model does not account for the structures required by the analysis model, the resulting data will lack utility.

Key Insights and Results

The Utility Trade-off

The study revealed a significant "tug-of-war" between statistical replication and temporal fidelity. The "Base" simulation (Condition 1) was highly effective at replicating the original study's p-values and regression coefficients. However, it failed to capture the "look and feel" of individual player trajectories over time. Conversely, when time-lags were introduced (Condition 4) to make individual load trends more realistic, the specific utility for the injury-prediction model decreased, causing results to diverge from the original findings.

Deterministic Relationships

A critical finding involved derived variables. In sports science, variables like "Chronic Load" are mathematically calculated from "Acute Load." The researchers found that simulating both as independent variables often broke the mathematical rules of the data. The study recommends simulating only the base variable and then calculating the derived variable within the synthetic set to maintain integrity.

Performance and Scalability

Global utility remained consistently high across all tests (pMSE < 0.01). However, computational bottlenecks were significant. Synthesizing variables with high cardinality (like PlayerID) is resource-intensive; one simulation required over 22 minutes for a single run, suggesting that large-scale iterations may require high-performance computing environments.

Discussion and Future Directions

While sequential tree-based algorithms are powerful, they are not a "silver bullet." Synthetic data should be viewed as a tool for data exploration and code sharing rather than a direct replacement for real data in final confirmatory analyses.

Future research should focus on:

  • Hierarchical Structures: Better handling of measures nested within players.
  • Standardized Documentation: Developing "Data Sheets" for synthetic sets to inform users of the specific "recipe" used and its inherent limitations.
  • Collaborative Pipelines: Encouraging closer cooperation between sports scientists and data scientists to ensure synthetic datasets are fit-for-purpose before public release.

By embracing synthetic data with a clear understanding of its boundary conditions, the sports science community can move closer to the goals of Open Science without compromising the privacy of the athletes it studies.

Summary updated

Abstract

Synthetic data represent alternative data sources generated using mathematical procedures to address specific issues in research and practice.Synthetic data has emerging applications in clinical and medical data contexts and may assist in overcoming privacy issues to help support open science practice.The present study discusses the applicability of an established synthetic data generation process using sequential tree-based algorithms (Synthpop package in R) in the context of athlete monitoring data in sport.We provide an educational primer and discussion for potential application of these methods when exploring issues in the field sports and exercise sciences via the application of Synthpop in seven simulation examples applied to a professional football dataset.Although sequential tree-based algorithms can create synthetic data using our reference dataset, we provide considerations for and highlight limitations when constructing synthetic data.To summarize, three types of models can be conceptualised for generating synthetic data: 1) models used for analysis of the original data (answering specific research questions); 2) models used to generate synthetic data, and; 3) models that represent the true generation process for the original data.Misalignments in the specifications of these models might introduce biases that can compromise the utility of synthetic data no matter the purpose.As synthetic data do not constitute a direct replacement of real data from conceptual and empirical standpoints, we believe that researchers embracing this practice must include sufficient documentation concerning the synthetic data generation process purpose, the predictors and model used, and the potential boundary conditions for using the synthetic data in future investigations in sports and other fields.
BibTeX
@misc{warmenhoven2024synthetic,
  author = {John Warmenhoven and Franco M. Impellizzeri and Ian Shrier and Andrew D. Vigotsky and Lorenzo Lolli and Paolo Menaspà and Aaron J. Coutts and Maurizio Fanchini and Giles Hooker},
  title = {{Synthetic data for sharing and exploration in high performance sport}},
  year = {2024},
  doi = {10.51224/srxiv.394},
}
Figure