Great GATsBi: Hybrid, Multimodal, Trajectory Forecasting for Bicycles using Anticipation Mechanism

Kevin Riehl, Shaimaa K. El-Baklish, Anastasios Kouvelas, Michail A. MakridisSubmission to ICLR 2026 · Rejected

TL;DR

Bicycles exhibit a unique "dual nature," combining vehicle-like kinematic constraints with pedestrian-like sudden maneuvers, making their trajectories notoriously difficult to predict. The authors introduce Great GATsBi, a hybrid, multimodal framework combining a physics-based ensemble (for short-term dynamics) with a psychology-inspired social Graph Attention Network (incorporating anticipation and memory decay for long-term interactions). Evaluated on a newly collected mass-cycling dataset and standard pedestrian benchmarks, the method significantly reduces forecasting error and uncertainty.

What Problem Does This Paper Address?

Vulnerable road users (VRUs), especially cyclists, make up a large share of traffic fatalities, yet most trajectory forecasting literature focuses heavily on pedestrians and motorized cars. Predicting cyclist trajectories is uniquely challenging:

  1. Like motor vehicles, bicycles have non-holonomic kinematic limits and reach moderate-to-high speeds.
  2. Like pedestrians, cyclists frequently deviate from strict lane boundaries and execute sudden direction changes.

Prior bicycle prediction methods relied either on pure data-driven deep learning or simplistic kinematic models, typically ignoring multimodality (predicting multiple plausible paths rather than a single trajectory) and treating neighborhood interactions with basic pooling or focal attention.

Methods

Great GATsBi integrates domain knowledge into a deep learning pipeline via four interconnected modules:

[ Historical Trajectory & Context ]
           │
     ┌─────┴──────────────┐
     ▼                    ▼
[ Physics Module ]   [ Social Module ]
  (Ensemble of 4       (GAT + Memory Decay +
   Kinematic Models)    Anticipated Futures)
     │                    │
     └─────┬──────────────┘
           ▼
   [ Fusion Module ] (LSTM)
           │
           ▼
   [ Output Module ] (GMM Decoder: unimodal or multimodal)
  • Physics Module: Leverages four classical kinematic models encoded via separate LSTMs into an ensemble embedding ($Y^{\text{Phy}}_i$):
  • Constant velocity (const v)
  • Constant acceleration (const a)
  • Single-track bicycle kinematics (kinematic)
  • Extended Kalman Filter (xkalman)
  • Social Module: Constructs a fully connected Graph Attention Network (GAT) to model human-human interactions, incorporating two psychology-informed concepts:
  • Perception Decay: Applies exponential decay weights ($D_h, D_p$) so timesteps nearest the current time exert stronger influence than distant past or future frames.
  • Neighborhood Anticipation: Predicts short-term future paths of surrounding neighbors (using const v) and passes both past and anticipated trajectories through LSTMs into the graph node embeddings. Edge features encode relative distance, angle, and speed.
  • Fusion Module: Concatenates physical and social latent representations and decodes them with an LSTM to produce a unified state $Z_i$.
  • Multimodal Decoder: Outputs parameters for a 2D Gaussian Mixture Model (GMM) ($\mu_x, \mu_y, \sigma_x, \sigma_y, \rho, \pi$) to capture trajectory multimodality, evaluated via expected trajectory sampling.

Datasets

  • Mass Cycling Experiment Dataset (New): A real-world, closed-loop dataset collected via aerial drone footage over a 5.0m-radius circular track with over 25 unique cyclists under varying densities (6–22 simultaneous riders, >16,000 frames). Designed to isolate physical dynamics and social interaction while controlling for complex intersection geometries.
  • Pedestrian Benchmarks: Standard ETH and HOTEL datasets, used to test how well the proposed social anticipation mechanism generalizes to pedestrian motion.

Key Insights and Ideas

  • Complementary Time Horizons: Physics-based baselines perform well over very short horizons (1s) where inertia dominates, but fail rapidly as time extends. Conversely, social interaction modeling dominates longer horizons (2–4s) where maneuvers like overtaking occur. Fusing both yields superior performance across all prediction lengths.
  • Anticipation Beats Static Observation: Explicitly estimating neighbors' near-future positions rather than relying solely on past trajectory history markedly improves GAT attention quality during multi-agent interactions.
  • Simplicity in Kinematics Wins: Among standalone physics baselines, the simple constant-velocity model (const v) outperformed more complex acceleration or kinematic models on average, motivating its use as the default forward-projection tool for neighboring agents.

Key Results

On the Mass Cycling Dataset (evaluating Average Displacement Error, ADE, and Final Displacement Error, FDE in meters):

  • Full Model Superiority: Great GATsBi achieved 0.0715m ADE / 0.1893m FDE at 1s, and 0.7543m ADE / 2.3827m FDE at 4s, outperforming baselines like SocialLSTM (0.8214m ADE / 2.3770m FDE at 4s) and Social-BiGAT (0.8069m ADE / 2.5356m FDE at 4s).
  • Physics Fusion Gains: Combining all four physics models improved performance by 4–6% compared to any sub-combination.
  • Uncertainty Reduction: Qualitative and quantitative distributions confirmed that the hybrid ensemble substantially tightened the predicted GMM probability distribution, minimizing variance and reducing horizontal drift bias compared to SocialLSTM and Social-BiGAT.
  • Pedestrian Generalization: On ETH/HOTEL, GATsBi remained competitive (second only to SocialLSTM), demonstrating the social module's cross-domain robustness.

Limitations and Future Work

  • Simplified Physics in Pedestrian Testing: The pedestrian evaluation reused bicycle kinematic components; incorporating pedestrian-specific models would provide a cleaner benchmark.
  • Controlled Environment Constraints: The experimental dataset used a circular track free of static obstacles, lane markings, or mixed motorized traffic.
  • High-Level Social Cognition: The current framework does not explicitly model rider aggressiveness, group riding dynamics, or intentions.
  • Architecture and Perception Extensions: Preliminary exploration of Transformer encoders showed negligible gains on small datasets, but the authors highlight Vision-Language Models (VLMs) and sequential physics-social architectures as promising future directions.

Summary updated

Abstract

Accurate prediction of road user movement is increasingly required by many applications ranging from advanced driver assistance systems to autonomous driving, and especially crucial for road safety. Even though most traffic accident facilities account to bicycles, they have received little attention, as previous work focused mainly on pedestrians and motorized vehicles. In this work, we present the Great GATsBi, a domain-knowledge-based, hybrid, multimodal trajectory prediction framework for bicycles. The model incorporates both physics-based modeling (inspired by motorized vehicles) and social-based modeling (inspired by pedestrian movements) to explicitly account for the dual nature of bicycle movement. The social interactions are modeled with a graph attention network, and include decayed historical, but also anticipated, future trajectory data of a bicycles neighborhood, following recent insights from psychological and social studies. The results indicate that the proposed ensemble of physics models - performing well in the short-term predictions - and social models - performing well in the long-term predictions - exceeds state-of-the-art performance. We also conducted a controlled mass-cycling experiment to demonstrate the framework's performance when forecasting bicycle trajectories and modeling social interactions with road users.
BibTeX
@misc{riehl2025great,
  author = {Kevin Riehl and Shaimaa K. El-Baklish and Anastasios Kouvelas and Michail A. Makridis},
  title = {{Great GATsBi: Hybrid, Multimodal, Trajectory Forecasting for Bicycles using Anticipation Mechanism}},
  year = {2025},
  eprint = {2508.14523},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.14523},
}
Figure