SRA: Spatial Reasoning Adapter via Evolving Social Interaction Graphs for Trajectory Prediction

Jaewoo Jeong, Seonkyu Song, Hyeonwoo Park, Jegyeong Cho, Yujin Bae, Giwon Lee, Daehee Park, Kuk-Jin YoonarXiv 2026 · arXiv

TL;DR

SRA (Spatial Reasoning Adapter) is a modular graph denoiser that improves multi-agent trajectory prediction by re-evaluating agent interactions at every step of the generative process. By treating denoising as progressive spatial reasoning over an evolving future, it ensures that predicted paths remain globally consistent and physically plausible across various model architectures.

What Problem Does This Paper Address?

Predicting how multiple agents (like cars, pedestrians, or athletes) move is a cornerstone of robotics and autonomous systems. Modern stochastic predictors, particularly those based on diffusion or flow-matching, typically analyze inter-agent interactions once based on the observed past and then keep that view fixed during the entire generation process.

The authors argue this is a major structural limitation. Critical interactions—such as a defender deciding which attacker to mark or two cars negotiating a narrow gap—often only emerge as the "joint future" takes shape. If these interactions aren't re-examined as the prediction sharpens, the resulting trajectories might be individually realistic but globally inconsistent, leading to "ghost" collisions or incoherent group behaviors.

Methods

SRA is designed as a "plug-and-play" adapter that can be integrated into existing iterative stochastic predictors (referred to as "hosts") without redesigning their generative cores. At every reverse denoising step, SRA reads the host's current "best guess" of the future and applies a residual correction.

The adapter consists of three core components:

  1. Sparse Neighborhood Selection: To avoid the quadratic cost and noise of a fully connected graph, SRA uses a learned query-key score combined with a geometric bias to select only the most relevant neighbors for each agent.
  2. Agent-Pair Relational Future Encoding: It builds edge features directly from the evolving future estimate, capturing relative positions and headings in an agent-centric frame.
  3. Uncertainty-Weighted Message Passing: SRA predicts a per-agent uncertainty. During message passing, more "confident" agents act as anchors, exerting a stronger influence on the refinement of their "uncertain" neighbors.
Overview of SRA at one reverse step

Datasets

The authors evaluate SRA on four multi-agent benchmarks:

  • NBA, Soccer, and Football: These sports datasets are particularly challenging due to dense, fast, and highly coupled interactions where future-conditioned coherence is vital.
  • SDD (Stanford Drone Dataset): A standard benchmark for pedestrian trajectory prediction.

Key Insights and Ideas

  • Denoising as Progressive Reasoning: The central "aha" moment is that the denoising chain in generative models is an ideal vehicle for structured reasoning. As noise is removed, the model can measure Euclidean distances and relative motions more accurately, allowing it to enforce social coherence "on the fly."
  • Uncertainty as a Functional Signal: Uncertainty isn't just an output for evaluation; it's used internally to route information. By letting confident agents "anchor" the scene, the model prevents early-stage noisy estimates from derailing the entire joint prediction.
  • Broad Compatibility: SRA was successfully instantiated on three structurally different hosts: LED (leapfrog DDPM), MID (vanilla DDPM), and MoFlow (flow-matching). This suggests that explicit future-future interaction is a fundamental missing piece in many current trajectory prediction paradigms.

Key Results

  • Quantitative Gains: SRA improved performance for every host on every dataset. On average, it achieved a 5.8% improvement in Average Displacement Error (ADE) and a 7.0% improvement in Final Displacement Error (FDE). The most significant gains were on complex sports scenes, reaching up to 18% ADE improvement.
  • Qualitative Coherence: Visualizations demonstrate that SRA-equipped models better capture tight couplings. For example, in basketball scenes, the ball's trajectory correctly tracks its carrier rather than drifting independently, and defenders correctly mirror an attacker's predicted run.

Main quantitative results across datasets Qualitative comparison on NBA scenes showing SRA's ability to capture tight couplings

Limitations and Future Work

  • Training Requirement: While modular, SRA is not "training-free." It requires a host-specific injection point and must be fine-tuned or jointly trained with the host model.
  • Inference Latency: Rebuilding the interaction graph at every step adds computational overhead. While the authors' optimized implementation stays within real-time budgets for 5Hz data (adding ~1.5–8.6 ms per scene), the $O(N^2)$ retrieval score could become a bottleneck for extremely large crowds.
  • Scene Scope: The current evaluation focuses on single-class scenes (e.g., only players or only pedestrians). Extending SRA to mixed-class scenes with heterogeneous dynamics and map-dependent interactions (like traffic rules) remains a direction for future work.

Summary updated

Abstract

Predicting where multiple agents will move next is fundamental to autonomous driving, robotics, and any system that must share space with people safely. Modern stochastic predictors read off inter-agent interactions once from the observed past and keep that view fixed throughout the generative rollout, so the reciprocal geometric constraints that emerge as the joint future sharpens are not re-examined. We address this gap with SRA (Spatial Reasoning Adapter), a modular graph denoiser that is integrated into existing diffusion and flow-matching predictors without redesigning their generative cores. At every reverse step, SRA reads the host's current future estimate, builds a sparse Euclidean interaction graph on the predicted trajectories, and returns a residual correction to the host's decoding path. Our adapter consists of three core components: a sparse neighborhood retrieved with a learned query-key score and a geometric bias, a future-conditioned relational feature recomputed on the evolving prediction, and uncertainty-weighted message passing where confident agents act as anchors for uncertain ones. We instantiate SRA on three structurally different hosts (LED, MID, and MoFlow) across four multi-agent benchmarks: SDD, NBA, Soccer, and Football. SRA improves ADE for every host on every dataset, with average gains of 5.8% in ADE and 7.0% in FDE. A cumulative ablation shows that the three components act as one mechanism: only together do they improve every host on every sports dataset, indicating that spatial reasoning over the model's own evolving future, weighted by how far that future can be trusted, is broadly useful across denoising architectures. Code is available at https://github.com/Jaewoo97/SRA.
BibTeX
@misc{jeong2026sra,
  author = {Jaewoo Jeong and Seonkyu Song and Hyeonwoo Park and Jegyeong Cho and Yujin Bae and Giwon Lee and Daehee Park and Kuk-Jin Yoon},
  title = {{SRA: Spatial Reasoning Adapter via Evolving Social Interaction Graphs for Trajectory Prediction}},
  year = {2026},
  eprint = {2611.10506},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10506},
}

Loading related content...

Figure