Abstract
Trajectory prediction for scenes with multiple agents and entities is a challenging problem in numerous domains such as traffic prediction, pedestrian tracking and path planning. We present a general architecture to address this challenge which models the crucial inductive biases of motion, namely, inertia, relative motion, intents and interactions. Specifically, we propose a relational model to flexibly model interactions between agents in diverse environments. Since it is well-known that human decision making is fuzzy by nature, at the core of our model lies a novel attention mechanism which models interactions by making continuous-valued (fuzzy) decisions and learning the corresponding responses. Our architecture demonstrates significant performance gains over existing state-of-the-art predictive models in diverse domains such as human crowd trajectories, US freeway traffic, NBA sports data and physics datasets. We also present ablations and augmentations to understand the decision-making process and the source of gains in our model.
Introduction
Predicting the future path of an agent—whether a pedestrian, a vehicle, or a charged particle—requires understanding its environment and the agents within it. In multi-agent settings, trajectories are rarely independent; they are shaped by complex, often subtle interactions. Traditional approaches often struggle to capture the nuance of these interactions, typically modeling them as discrete graph edges or using standard attention mechanisms that lack explicit decision-making logic. This work introduces Fuzzy Query Attention (FQA), a novel relational model that treats interactions as continuous-valued "fuzzy" decisions, mirroring the nuanced nature of human decision-making.
Methodology
The proposed architecture is built upon four fundamental inductive biases of motion:
- Inertia: The tendency of objects to maintain constant velocity.
- Relative Motion: The principle that observations and predictions should be relative to an agent's current state.
- Intent: Internal goals, modeled via an LSTM, which drive deviations from inertial motion.
- Interactions: External influences exerted by other agents in the scene.
The core of the interaction component is the Fuzzy Query Attention (FQA) module. Unlike standard multi-head self-attention, FQA explicitly models pairwise "sender-receiver" interactions. It produces a "fuzzy decision" value $D_{sr} \in [0, 1]$ using a dot-product structure with a learnable bias term $B$ and a sigmoid activation. This decision value governs a "fuzzy if-else" logic, where the model learns and combines two separate responses—a "yes" response and a "no" response—allowing for flexible transitions between different interaction behaviors.
Experimental Results
The FQA model was evaluated across five diverse datasets: ETH-UCY (pedestrians), Collisions (synthetic physics), NGsim (freeway traffic), Charges (electromagnetic particles), and NBA (sports trajectories).
FQA consistently outperformed state-of-the-art baselines, including Social LSTM, NRI, and Graph Attention Networks (GAT). Key findings include:
- Physics Modeling: On the "Collisions" dataset, FQA achieved a nearly 25% reduction in RMSE compared to the next best model. Remarkably, the model's fuzzy decisions were over 95% accurate at predicting collision events without explicit supervision.
- Traffic Prediction: In the NGsim dataset, FQA achieved an RMSE of 5.071, significantly better than Vanilla LSTM (5.972) and GAT (6.100).
- Ablation Success: Removing the fuzzy decision component led to performance drops down to baseline levels, confirming the structural importance of the fuzzy logic mechanism.
Conclusion
Fuzzy Query Attention offers a robust and interpretable framework for multi-agent trajectory prediction. By explicitly modeling interactions as continuous decisions, the architecture achieves superior performance across a wide range of domains. While challenges remain in high-intent environments like professional basketball, the model's ability to learn underlying interaction physics without explicit supervision marks a significant advancement. Future work will explore the integration of visual data to better capture high-level intent in complex scenarios.
Summary updated