Abstract
Pedestrian trajectory prediction is critical in many vision tasks but challenging due to the multimodality of the future trajectory. Most existing methods predict multi-modal trajectories conditioned by goals (future endpoints) or instances (all future points). However, goal-conditioned methods ignore the intermediate process and instance-conditioned methods ignore the stochasticity of pedestrian motions. In this paper, we propose a simple yet effective Sparse Instance Conditioned Network (SICNet), which gives a balanced solution between goal-conditioned and instance-conditioned methods. Specifically, SICNet learns comprehensive sparse instances, i.e., representative points of the future trajectory, through a mask generated by a long short-term memory encoder and uses the memory mechanism to store and retrieve such sparse instances. Hence SICNet can decode the observed trajectory into the future prediction conditioned on the stored sparse instance. Moreover, we design a memory refinement module that refines the retrieved sparse instances from the memory to reduce memory recall errors. Extensive experiments on ETH-UCY and SDD datasets show that our method outperforms existing state-of-the-art methods. In addition, ablation studies demonstrate the superiority of our method compared with goal-conditioned and instance-conditioned approaches.
Introduction
Predicting where a pedestrian will walk is notoriously difficult because human movement is inherently multimodal. Given a specific past path, a person might turn left, continue straight, or stop abruptly. Current AI models generally fall into two categories: goal-conditioned and instance-conditioned.
Goal-conditioned models predict a final destination and then interpolate the path. While simple, they often ignore the nuances of the intermediate journey. Conversely, instance-conditioned models use entire future trajectories as templates. While detailed, these are computationally expensive and often struggle with the natural stochasticity of human motion. SICNet introduces a "Goldilocks" approach by using sparse instances—representative points along a future path—to provide guidance that is more descriptive than a single goal but more flexible than a full trajectory.
Methodology
SICNet operates via a two-stage strategy involving sparse representation and an external memory mechanism.
Sparse Instance Representation
Instead of using every point in a future trajectory, the model learns a "mask" to extract a sparse set of representative features. This captures the essence of the movement without the overhead of a full path. This representation allows the decoder to reconstruct a realistic path while maintaining the flexibility to handle motion randomness.
Memory Mechanism and Refinement
The model utilizes an external memory bank to store pairs of past trajectory features and their corresponding sparse future instances. During inference, the model observes the past and "recalls" the most relevant future patterns.
A key innovation is the Memory Refinement Module. Since real-world observations rarely match stored memories perfectly, this module calculates the deviation between the current observation and the retrieved memory, refining the sparse instance to better fit the current context. Finally, to provide a diverse set of $K$ predictions, the model clusters retrieved candidates to remove redundant proposals.
Experiments and Results
The model was evaluated on two gold-standard benchmarks: ETH-UCY (real-world pedestrian coordinates) and the Stanford Drone Dataset (SDD) (large-scale bird's-eye view trajectories).
SICNet outperformed existing state-of-the-art (SOTA) methods across nearly all metrics:
- ETH-UCY: Achieved an Average Displacement Error (ADE) of 0.19 and a Final Displacement Error (FDE) of 0.33, surpassing previous leaders like MemoNet and PECNet.
- SDD: Achieved an ADE of 8.44 and FDE of 13.65. Notably, SICNet was particularly effective at predicting the "in-between" states of a trajectory where goal-conditioned methods typically fail.
- Efficiency: The sparse representation allowed the model to maintain high accuracy even when predicting a small number of potential futures ($K=5$), demonstrating better adaptability than rigid instance-based models.
Conclusion
SICNet demonstrates that sparsity is a powerful tool for trajectory prediction. By focusing on representative waypoints rather than full paths or single goals, the model acts like an experienced observer that adapts past "experiences" to new realities. While the current model achieves SOTA results without explicit social or map context, future work could integrate these data streams to further enhance safety-critical applications like autonomous driving.
Summary updated