Abstract
To comprehensively cover the uncertainty of the future, the common practice of multi-modal human trajectory prediction is to first generate a set/distribution of candidate future trajectories and then sample required numbers of trajectories from them as final predictions. Even though a large number of previous researches develop various strong models to predict candidate trajectories, how to effectively sample the final ones has not received much attention yet. In this paper, we propose stimulus verification, serving as a universal and effective sampling process to improve the multi-modal prediction capability, where stimulus refers to the factor in the observation that may affect the future movements such as social interaction and scene context. Stimulus verification introduces a probabilistic model, denoted as stimulus verifier, to verify the coherence between a predicted future trajectory and its corresponding stimulus. By highlighting prediction samples with better stimulus-coherence, stimulus verification ensures sampled trajectories plausible from the stimulus' point of view and therefore aids in better multi-modal prediction performance. We implement stimulus verification on five representative prediction frameworks and conduct exhaustive experiments on three widely-used benchmarks. Superior results demonstrate the effectiveness of our approach.
Introduction
Predicting human movement is fundamentally uncertain; a pedestrian might turn, stop, or continue straight. Modern "multi-modal" AI models address this by generating a cloud of potential future paths. However, the final step—selecting the most plausible $k$ paths—is often overlooked, typically relying on simple random or Monte-Carlo sampling. This neglect often results in "impossible" predictions that violate physical constraints (e.g., walking through walls) or social etiquette (e.g., colliding with others).
This paper introduces Stimulus Verification, a universal, plug-and-play sampling module. It acts as an intelligent filter, scoring candidate paths based on their coherence with environmental "stimuli" like scene context and social interactions.
Methodology
The core component is the Stimulus Verifier ($V$), a probabilistic model that evaluates the likelihood of a trajectory given specific environmental cues.
The Stimulus Verifier
Unlike a binary classifier, the verifier employs Conditional Parameterization. It processes a stimulus (e.g., a semantic map) through a neural network to generate parameters for a Gaussian Mixture Model (GMM). This GMM then computes the exact likelihood that a specific candidate path is coherent with the observed stimulus.
Types of Stimuli
- Scene Context: A CNN processes semantic maps using a "sliding window" approach, cropping the map around the agent's predicted position to maintain local context over time.
- Social Interaction: A Graph Convolutional Network (GCN) models the spatial relationships between nearby agents, while an LSTM aggregates these social cues temporally.
Diversity-Aware Sampling
To ensure the final predictions are not redundant, the authors implement a Non-Maximum Suppression (NMS) strategy. The highest-scoring path is selected first, and subsequent candidates that are too spatially similar are discarded, ensuring a set that is both realistic and diverse.
Experimental Results
The method was evaluated on three benchmarks: ETH/UCY (outdoor plazas), Grand Central Station (GCS) (complex indoor crowds), and NBA SportVU (high-speed basketball movements).
Key Findings
- Universal Improvement: Stimulus Verification consistently boosted the performance of five distinct base models, including Social GAN and PCCSNet. On ETH/UCY, accuracy improved by up to 15.9%.
- Top-1 Precision: In the GCS dataset, the verifier improved Top-1 Average Displacement Error (ADE) by 17.8% and Final Displacement Error (FDE) by 19.4%, demonstrating its ability to identify the single most likely path.
- Model Agnosticism: The verifier functions as an external "critic," allowing it to be integrated into various architectures (GANs, VAEs, etc.) without retraining the base model.
Conclusion
Stimulus Verification shifts the focus from trajectory generation to effective sampling. By verifying candidate paths against physical and social constraints, it significantly enhances the reliability of multi-modal predictions. While it introduces some computational overhead, its plug-and-play nature and consistent performance gains across diverse datasets make it a powerful tool for autonomous systems and crowd analysis. Future work may involve deeper integration of these verification principles directly into the training phase of trajectory models.
Summary updated