Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling

Keisuke Fujii, Naoya Takeishi, Yoshinobu Kawahara, Kazuya Takeda

Introduction

Modeling group dynamics in high-stakes, high-speed environments—such as professional basketball and soccer—presents a significant challenge for artificial intelligence. While modern data-driven models excel at pattern recognition, they often fail to capture the underlying biological and physical realities of human movement. This research introduces a decentralized imitation learning framework designed to bridge the gap between pure data-driven prediction and biological plausibility by enforcing partial observation and mechanical constraints.

The Limitations of Current Multi-Agent Models

Most contemporary models for multi-agent trajectory prediction treat participants as "black boxes" operating with perfect information. These models typically assume that every agent possesses a global view of the field, tracking all other players and the ball simultaneously. Furthermore, they often ignore the laws of physics, allowing agents to change direction or velocity instantaneously without regard for inertia.

In reality, human athletes are subject to two primary limitations:

  1. Partial Observation: An athlete cannot process the movements of 21 other players and a ball at once. Instead, they selectively filter for the most relevant environmental cues.
  2. Mechanical Constraints: Human movement is governed by biomechanics. We cannot "teleport" or achieve infinite acceleration; our paths are naturally smooth and constrained by physical effort.

Ignoring these factors results in models that lack both interpretability (it is unclear why a player moved if we do not know what they saw) and plausibility (predicted paths often appear jittery or physically impossible).

Methodology: A Decentralized POMDP Approach

The researchers formulated the problem as a decentralized Partially Observable Markov Decision Process (POMDP). By treating each player as an independent agent with limited information, the model more closely mimics real-world decision-making. The framework introduces three key technical innovations:

1. Binary Partial Observation Model

Rather than using standard "soft attention" mechanisms—which assign a small weight to every entity in the environment—the authors utilized a Gumbel-softmax reparameterization to create a "multi-hot" binary vector. This forces the model to make explicit choices about which specific teammates or opponents to track. This sparsity makes the model’s "vision" highly interpretable, allowing researchers to see exactly which cues triggered a specific movement.

2. Mechanical Constraint Penalties

To ensure physical realism without the rigidity of hard mathematical constraints (which often fail when applied to noisy real-world tracking data), the authors introduced "soft" penalties into the loss function:

  • Minimum-Jerk Principle: Derived from neuroscience, this principle suggests that humans plan movements to minimize the rate of change of acceleration, resulting in smooth, fluid trajectories.
  • Physical Consistency: This penalty ensures that the predicted position, velocity, and acceleration remain mathematically consistent over time, preventing the "jitter" common in standard neural network outputs.

3. Hierarchical VRNN with Macro Goals

The model employs a hierarchical Variational Recurrent Neural Network (VRNN). A high-level "macro goal" component handles long-term intent (e.g., a player deciding to sprint to the three-point line), while the low-level VRNN manages the frame-by-frame "micro" movements required to reach that goal.

Experimental Validation

The model was validated using two high-fidelity datasets:

  • NBA Tracking Data (2015-2016): 100 games comprising nearly 20,000 sequences, focusing on how defensive players react to offensive maneuvers.
  • Professional Soccer Data: Trajectories from 45 professional matches. This dataset provided a more rigorous test due to the larger pitch and the increased complexity of managing 22 agents.

Key Insights and Results

The study yielded several critical findings regarding the intersection of physics and machine learning:

  • Physics as a Regularizer: Adding physical constraints did more than just improve the visual quality of the trajectories; it acted as a regularizer that helped the model learn more efficiently. By penalizing physically impossible movements, the search space for the neural network was narrowed to realistic behaviors.
  • Interpretability through Sparsity: The binary observation mask allowed for the visualization of defensive focus. For instance, the model naturally learned to focus on the ball carrier and the direct opponent, aligning with established coaching principles.
  • Counterfactual Simulation: The decentralized nature of the model enables "what-if" scenarios. Researchers can manually alter a player's observation mask—for example, forcing a defender to ignore the ball—to simulate how a beginner's mistake might propagate through the rest of the play.
  • Performance and Smoothness: The proposed model significantly reduced positional and acceleration errors compared to standard VRNNs. Qualitatively, the trajectories mimicked the smooth acceleration profiles of elite athletes, avoiding the unrealistic velocity spikes seen in baseline models.

Conclusion and Future Directions

By incorporating biological and physical priors, this framework produces multi-agent simulations that are both more accurate and more interpretable than traditional black-box models. However, several avenues for future research remain. The current model lacks a "ground truth" for vision, as eye-tracking data is not available in standard sports datasets. Additionally, the model does not yet account for visuomotor delay (the latency between stimulus and reaction) or physiological fatigue, both of which are vital for professional-grade sports analytics. Finally, while the decentralized approach excels at individual realism, capturing the highly synchronized "set plays" of a cohesive team remains a complex challenge for future iterations.

Summary updated

Abstract

Extracting the rules of real-world multi-agent behaviors is a current challenge in various scientific and engineering fields. Biological agents independently have limited observation and mechanical constraints; however, most of the conventional data-driven models ignore such assumptions, resulting in lack of biological plausibility and model interpretability for behavioral analyses. Here we propose sequential generative models with partial observation and mechanical constraints in a decentralized manner, which can model agents’ cognition and body dynamics, and predict biologically plausible behaviors. We formulate this as a decentralized multi-agent imitation-learning problem, leveraging binary partial observation and decentralized policy models based on hierarchical variational recurrent neural networks with physical and biomechanical penalties. Using real-world basketball and soccer datasets, we show the effectiveness of our method in terms of the constraint violations, long-term trajectory prediction, and partial observation. Our approach can be used as a multi-agent simulator to generate realistic trajectories using real-world data.
BibTeX
@article{fujii2024decentralized,
  author = {Keisuke Fujii and Naoya Takeishi and Yoshinobu Kawahara and Kazuya Takeda},
  title = {{Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling}},
  journal = {Neural Networks},
  year = {2024},
  doi = {10.1016/j.neunet.2023.11.068},
}
Figure