What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction

Christoph Schöller, Vincent Aravantinos, Florian Lay, Alois C. Knoll

Abstract

In the pursuit of high-performance pedestrian motion prediction, the research community has increasingly turned toward complex neural network architectures, such as LSTMs, GANs, and attention mechanisms. This paper challenges the necessity of such complexity by demonstrating that a simple Constant Velocity Model (CVM)—which assumes a pedestrian maintains their current speed and direction—frequently outperforms state-of-the-art deep learning models. Our analysis reveals that many complex models fail to generalize because they overfit to scene-specific environmental priors and struggle to extract meaningful signals from social interactions or long-term history.

Introduction

Predicting pedestrian trajectories is a critical task for autonomous systems. While modern deep learning approaches aim to capture intricate social dynamics and historical patterns, they often exhibit a "simplicity paradox": despite their theoretical sophistication, they struggle to beat basic physics-based baselines in real-world benchmarks. This study investigates why these complex models underperform and what they are actually learning from the data.

Methodology

The authors evaluate several models against a baseline Constant Velocity Model (CVM). The primary variants include:

  • CVM (OUR): A deterministic extrapolation of the last observed velocity.
  • OUR-S: A stochastic version of the CVM that introduces random angular noise to generate a distribution of possible future paths, facilitating a fair comparison with generative models like GANs.

These are compared against standard baselines (Linear Regression, LSTMs) and state-of-the-art (SOTA) models including Social GAN (S-GAN), SoPhie, and RED across the ETH and UCY datasets.

Challenging Common Assumptions

Environmental Overfitting

Neural networks often inadvertently learn "environmental priors"—specific layout shortcuts of the training scene (e.g., a specific sidewalk's orientation). When tested on new environments, these models fail. The authors demonstrate that using relative coordinates and random rotations during training is essential to force models to learn motion dynamics rather than static scene geometry.

The "History" Myth

It is a common belief that longer historical trajectories lead to better future predictions. However, the authors found that 80% of a model's prediction is determined by the single most recent timestep. Analysis of motion correlation shows that most historical data is redundant.

Correlation matrices for X and Y motion across timesteps, illustrating high redundancy in pedestrian history.

Interactions as Noise

Modeling social interactions—how pedestrians avoid one another—is a core focus of SOTA models. Surprisingly, the authors found that providing neighbor information often acted as noise rather than a helpful signal. In many instances, models without social awareness performed better than those explicitly designed to account for it.

Key Results

The quantitative evaluation provided a significant wake-up call for the field. The noisy CVM (OUR-S) achieved the lowest Average Displacement Error (ADE) of 0.28m, outperforming complex models like S-GAN (0.32m) and SoPhie (0.54m).

Comparison of predicted trajectories between the noisy CVM (OUR-S) and complex models like S-GAN and RED.

In the challenging "Hotel" scene, standard neural networks initially failed with an ADE of 1.59m due to overfitting. By applying the "Relative + Rotation" fix, the error dropped to 0.30m, highlighting that the failure was due to poor generalization rather than model capacity.

Conclusion

The success of the Constant Velocity Model suggests that current deep learning approaches for pedestrian prediction are not yet effectively capturing the nuances they were designed for. Future research should prioritize robust generalization through environmental semantics and better noise filtering for social interactions. The CVM should serve as the fundamental baseline for all future work in this domain.

Summary updated

Abstract

Pedestrian motion prediction is a fundamental task for autonomous robots and vehicles to operate safely. In recent years many complex approaches based on neural networks have been proposed to address this problem. In this work we show that - surprisingly - a simple Constant Velocity Model can outperform even state-of-the-art neural models. This indicates that either neural networks are not able to make use of the additional information they are provided with, or that this information is not as relevant as commonly believed. Therefore, we analyze how neural networks process their input and how it impacts their predictions. Our analysis reveals pitfalls in training neural networks for pedestrian motion prediction and clarifies false assumptions about the problem itself. In particular, neural networks implicitly learn environmental priors that negatively impact their generalization capability, the motion history of pedestrians is irrelevant and interactions are too complex to predict. Our work shows how neural networks for pedestrian motion prediction can be thoroughly evaluated and our results indicate which research directions for neural motion prediction are promising in future.
BibTeX
@article{scholler2020what,
  author = {Christoph Schöller and Vincent Aravantinos and Florian Lay and Alois C. Knoll},
  title = {{What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2020},
  doi = {10.1109/lra.2020.2969925},
}
Figure