What the Constant Velocity Model Can Teach Us About Pedestrian Motion\n Prediction

Christoph Schöller, Vincent Aravantinos, Florian Lay, Alois Knoll

TL;DR

This paper reveals that a surprisingly simple Constant Velocity Model (CVM) can outperform complex state-of-the-art neural networks for pedestrian motion prediction. The authors analyze why this is the case, finding that neural networks often fail to effectively use additional information like environmental context and motion history, and that pedestrian interactions are less critical than commonly assumed.

What Problem Does This Paper Address?

Predicting the future movement of pedestrians is crucial for the safe operation of autonomous systems like robots and self-driving cars. As pedestrians are unpredictable agents, accurately forecasting their paths is a significant challenge. In recent years, researchers have increasingly turned to sophisticated neural network architectures, hoping they can leverage rich contextual information (like scene layout and the movements of other people) to achieve better predictions. However, the authors question whether these complex models are truly outperforming simpler approaches, or if there are fundamental issues with how these networks are trained and what information they actually utilize.

Methods

The core of the paper is a comparative evaluation. The authors first establish a baseline using a Constant Velocity Model (CVM), which assumes a pedestrian will continue moving in their current direction and at their current speed. They then compare this CVM against several other models, including:

  • Baselines: Constant Acceleration (ConstAcc) and Linear Regression (Lin).
  • Feed Forward Neural Network (FF): A standard fully connected network.
  • LSTM Network (LSTM): A recurrent neural network designed for sequential data.
  • State-of-the-Art Models: Including RNN-Encoder-MLP (RED), SR-LSTM, Social GAN (S-GAN), and SoPhie.

To understand why the CVM performs well, the authors conduct a series of analyses:

  • Environmental Priors: They investigate if neural networks implicitly learn biases from the scene's layout, even without explicit environmental input. They test this by training models with absolute positions versus relative motion, and by adding random rotations to training data to break directional biases.
  • Motion History: They analyze the influence of past timesteps on predictions by computing gradient norms and linear correlation coefficients between historical positions. They also test models with progressively shorter motion histories.
  • Pedestrian Interactions: They compare models trained with no neighbor information, with neighbor histories, and with perfect knowledge of neighbor futures to assess the impact of interactions.

Datasets

The experiments were conducted on two widely used pedestrian motion prediction datasets:

  • ETH dataset: Includes ETH-Uni and Hotel scenes.
  • UCY dataset: Includes Zaral, Zara2, and UCY-Uni scenes.

These datasets consist of real-world video recordings from various scenarios like university campuses and pedestrian zones, featuring non-traversable obstacles and diverse walkway layouts. The authors used a leave-one-out cross-validation approach, training on four scenes and evaluating on the remaining one to assess generalization capabilities.

Key Insights and Ideas

  • Simplicity Wins (Sometimes): The most striking finding is that a simple CVM, which only uses the last two observed positions, can outperform complex, state-of-the-art neural networks. This suggests that the additional information provided to neural networks might not be effectively utilized or is less relevant than assumed.
  • Environmental Priors are a Double-Edged Sword: Neural networks tend to learn implicit "environmental priors" based on the scene's layout. While this can be helpful in familiar scenes, it significantly harms generalization to new environments with different typical motion patterns (e.g., vertical vs. horizontal movement). Data augmentation techniques can help mitigate this by preventing the learning of these strong priors.
  • Motion History is Largely Redundant: Contrary to common belief, longer motion histories don't significantly improve predictions. Neural networks primarily rely on the most recent timestep, with earlier history often acting as noise. Depriving networks of longer histories doesn't degrade performance.
  • Interactions are Overrated (for Prediction): Predicting pedestrian interactions is extremely complex. Providing models with neighbor histories or even perfect future neighbor trajectories doesn't consistently improve predictions and can sometimes be detrimental. The impact of interactions on individual trajectories might be less significant than assumed, or too complex for current models to reliably exploit.

Key Results

  • Quantitative: The CVM (and a slightly modified version, OUR-S) achieved the best Average Displacement Error (ADE) and Final Displacement Error (FDE) across multiple scenes, outperforming models like S-GAN and SoPhie. For instance, on average, OUR-S achieved an ADE of 0.28, significantly better than S-GAN (0.32) and SoPhie (0.39).
  • Qualitative: Visualizations show that the CVM's linear predictions are often good approximations of pedestrian paths, even for slightly curved trajectories. While it can't predict sudden sharp turns, its simplicity leads to robust performance. The analysis of gradient norms showed that the latest timestep accounts for ~68% of the influence for FF and ~80% for RED, with earlier timesteps contributing much less.

Limitations and Future Work

  • Limitations: The authors acknowledge that the CVM's linear nature means it cannot extrapolate non-linear behaviors like sudden turns. They also note that their analysis of interactions might be limited by the complexity of modeling them.
  • Future Work: The paper suggests a stronger focus on developing datasets with more diverse environments to improve generalization. It also proposes investigating whether the insights gained about environmental priors, motion history, and interactions are transferable to other motion prediction domains, such as vehicle-to-vehicle or vehicle-to-pedestrian prediction, where interactions might be more structured and predictable. The authors advocate for the CVM to be used as a standard baseline for future research.

Summary updated

Abstract

Pedestrian motion prediction is a fundamental task for autonomous robots and\nvehicles to operate safely. In recent years many complex approaches based on\nneural networks have been proposed to address this problem. In this work we\nshow that - surprisingly - a simple Constant Velocity Model can outperform even\nstate-of-the-art neural models. This indicates that either neural networks are\nnot able to make use of the additional information they are provided with, or\nthat this information is not as relevant as commonly believed. Therefore, we\nanalyze how neural networks process their input and how it impacts their\npredictions. Our analysis reveals pitfalls in training neural networks for\npedestrian motion prediction and clarifies false assumptions about the problem\nitself. In particular, neural networks implicitly learn environmental priors\nthat negatively impact their generalization capability, the motion history of\npedestrians is irrelevant and interactions are too complex to predict. Our work\nshows how neural networks for pedestrian motion prediction can be thoroughly\nevaluated and our results indicate which research directions for neural motion\nprediction are promising in future.\n
BibTeX
@article{scholler2020what,
  author = {Christoph Schöller and Vincent Aravantinos and Florian Lay and Alois Knoll},
  title = {{What the Constant Velocity Model Can Teach Us About Pedestrian Motion\textbackslash\{\}n Prediction}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2020},
  doi = {10.1109/lra.2020.2969925},
}
Figure