A Location-Velocity-Temporal Attention LSTM Model for Pedestrian Trajectory Prediction

Hao Xue, Du Q. Huynh, Mark Reynolds

Abstract

Pedestrian trajectory prediction is a cornerstone of autonomous navigation and surveillance. While modern approaches often rely on dense environmental context—such as maps and neighbor interactions—these methods suffer from high computational costs and poor generalizability. This paper introduces LVTA (Location-Velocity-Temporal Attention), a streamlined LSTM-based model that predicts future paths using only an individual's observed trajectory. By employing dual-stream LSTMs for location and velocity, dual temporal attention mechanisms, and a novel "tweak module," LVTA achieves state-of-the-art accuracy across major benchmarks while maintaining high efficiency and robustness in unseen environments.

Introduction: The Challenge of Context-Heavy Models

Predicting pedestrian movement is complex due to the inherent social and environmental factors influencing human behavior. Current advanced models typically incorporate "context-heavy" data, including the positions of nearby pedestrians and static obstacle maps. However, this dependency introduces significant drawbacks: high computational overhead for real-time processing, a requirement for perfectly labeled environmental data, and a tendency to overfit to specific training scenes.

The researchers behind LVTA propose a "less is more" philosophy. They argue that a pedestrian's own trajectory—specifically their velocity and historical movement—implicitly encodes environmental constraints. If a person slows down or turns, it reflects their response to an obstacle or another person, even if that obstacle isn't explicitly modeled.

The LVTA Architecture: A Streamlined Approach

The LVTA model is built upon three core technical innovations designed to extract maximum information from minimal data:

  1. Dual-Stream LSTMs: Unlike standard models that process only coordinates, LVTA uses two parallel Long Short-Term Memory (LSTM) networks. One stream processes raw $(x, y)$ coordinates (Location), while the other focuses on the change in position (Velocity). This allows the model to treat velocity as a primary indicator of intent.
  2. Temporal Attention Mechanisms: Standard LSTMs often rely on the final hidden state for prediction. LVTA applies temporal attention to the hidden state vectors of both the location and velocity streams. This mechanism weights historical moments differently, allowing the model to "remember" critical past maneuvers that influence future direction.
  3. The Tweak Module: Positioned between the observation and prediction phases, this module contains a location-velocity attention layer. It refines the predicted coordinates before they are passed to the next time step, ensuring that the output remains physically plausible.
Architecture of the LVTA model showing the dual-stream LSTM processing and attention mechanisms.

Experimental Evaluation and Results

The model was evaluated against eleven existing trajectory prediction methods using three large-scale benchmark datasets: Central Station, ETH & UCY, and the Edinburgh dataset.

Performance Metrics

LVTA demonstrated competitive and often superior performance compared to context-aware models:

  • Central Station Dataset: The model achieved an Average Displacement Error (ADE) of 9.19 pixels and a Final Displacement Error (FDE) of 17.28 pixels, outperforming models like Social-LSTM.
  • ETH & UCY Datasets: On these outdoor benchmarks, LVTA reached an average ADE of 0.46 meters and an FDE of 0.92 meters, placing it among the top-performing models globally.

Generalizability and Efficiency

A key strength of the LVTA model is its ability to generalize. Because it does not rely on scene-specific maps, it can be deployed in new environments without retraining. This was validated using the Edinburgh dataset, where the model successfully predicted trajectories in a completely new scene after being trained on the Central Station data. Furthermore, the absence of complex social pooling layers makes LVTA significantly faster to train and more suitable for real-time deployment on edge devices like drones or mobile robots.

Conclusion

The LVTA model proves that high-accuracy pedestrian prediction does not strictly require complex environmental context. By focusing on the intrinsic properties of an individual's movement through dual-stream LSTMs and attention mechanisms, LVTA offers a robust, efficient, and generalizable solution for trajectory forecasting. Future work will explore the joint training of these attention mechanisms and the application of LVTA in detecting abnormal behaviors in surveillance systems.

Reference: Xue, H., Huynh, D. Q., & Reynolds, M. (2020). A Location-Velocity-Temporal Attention LSTM Model for Pedestrian Trajectory Prediction. IEEE Access, 8, 44576-44589.

Summary updated

Abstract

Pedestrian trajectory prediction is fundamental to a wide range of scientific research work and industrial applications. Most of the current advanced trajectory prediction methods incorporate context information such as pedestrian neighbourhood, labelled static obstacles, and the background scene into the trajectory prediction process. In contrast to these methods which require rich contexts, the method in our paper focuses on predicting a pedestrian's future trajectory using his/her observed part of the trajectory only. Our method, which we refer to as LVTA, is a Location-Velocity-Temporal Attention LSTM model where two temporal attention mechanisms are applied to the hidden state vectors from the location and velocity LSTM layers. In addition, a location-velocity attention layer embedded inside a tweak module is used to improve the predicted location and velocity coordinates before they are passed to the next time step. Extensive experiments conducted on three large benchmark datasets and comparison with eleven existing trajectory prediction methods demonstrate that LVTA achieves competitive prediction performance. Specifically, LVTA attains 9.19 pixels Average Displacement Error (ADE) and 17.28 pixels Final Displacement Error (FDE) for the Central Station dataset, and 0.46 metres ADE and 0.92 metres FDE for the ETH&UCY datasets. Furthermore, evaluation on using LVTA to generate trajectories of different prediction lengths and on new scenes without the need of retraining confirms that it has good generalizability.
BibTeX
@inproceedings{xue2020location,
  author = {Hao Xue and Du Q. Huynh and Mark Reynolds},
  title = {{A Location-Velocity-Temporal Attention LSTM Model for Pedestrian Trajectory Prediction}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=4W09FYtBYg},
}
Figure