Abstract
For an autonomous car, getting surprised is the worst thing that can happen. To prevent that, plenty of studies are trying to forecast traffic participants’ actions especially in an urban scene using recurrent networks. Recurrent networks are used for temporal tasks in many application domains like Natural Language processing and computer vision. Despite the tendency in the literature to use recurrent neural networks for trajectory prediction, we argue that because of small dependency in trajectory sequences of a vehicle, a feed-forward neural network can be used, instead. In this paper, we will compare these two methods in vehicle trajectory prediction while considering the vanilla models or taking the scene into account. In order to have more variations in trajectories, roundabouts are used as a case study. Our results show that the proposed feed-forward network has competitive results with a recurrent network with significantly faster processing time.
Introduction: The Challenge of Urban Trajectory Prediction
For an autonomous vehicle, "getting surprised" is the ultimate failure. To navigate safely, a car must predict where other agents—cars, cyclists, and pedestrians—will be in the next few seconds. Traditionally, researchers have treated this as a sequence-to-sequence problem, naturally reaching for Recurrent Neural Networks (RNNs) or Long Short-Term Memory (LSTM) networks. The assumption is that because a trajectory is a series of points over time, you need a model with "memory."
However, RNNs are inherently sequential, making them difficult to parallelize and slow to run in real-time. This paper challenges the status quo, asking if the complex temporal machinery of an LSTM is actually necessary for the relatively short-term dependencies found in urban driving.
Methodology: Architecture and Scene Awareness
The authors compare two architectural philosophies using an encoder-decoder framework. The standard Recurrent (LSTM) approach processes past positions sequentially to build a hidden state. In contrast, the Feed-Forward (MLP) approach treats the entire history as a single input vector and predicts the entire future horizon simultaneously.
Both models are tested in "Vanilla" versions (using only past coordinates) and "Scene-aware" versions. The scene-aware models use a Convolutional Neural Network (CNN) branch to process a cropped, bird’s-eye view image of the road geometry. This allows the model to "see" the boundaries of the road, which is critical in complex environments like roundabouts.
Experimental Setup: The EPFL-Roundabout Dataset
The study introduces and utilizes the EPFL-Roundabout Dataset, featuring drone footage of four different roundabouts in Lausanne, Switzerland. The dataset includes over 2 hours of 25fps video, capturing more than 4,500 vehicles across five categories (cars, trucks, buses, pedestrians, and cyclists). The models were trained on three roundabouts and tested on two, including one completely "unseen" roundabout to test generalization.
Results: Accuracy vs. Efficiency
The results demonstrate that simple Feed-Forward (FF) networks can predict vehicle trajectories just as accurately as LSTMs while being significantly faster:
- Accuracy: On the "unseen" roundabout (Morges Avenue), the Scene-FF model achieved an Average Displacement Error (ADE) of 0.98m, nearly identical to the Scene-RNN at 0.97m.
- Speed: The Feed-Forward (MLP) model was significantly more efficient. The test time for the Scene-MLP was 0.0014 seconds, compared to 0.016 seconds for the Scene-LSTM—representing an 11.4x speedup.
- Qualitative Impact: Adding scene information drastically improved performance. Without the scene, models simply extrapolated straight lines; with the scene, they correctly predicted vehicles following the curve of the roundabout.
Discussion: Why Feed-Forward Works
The success of the FF model supports the "Short Memory" Hypothesis: Unlike natural language, where a word at the start of a sentence can change the meaning of the last word, a vehicle's trajectory is largely governed by its immediate momentum and the physical constraints of the road. This lack of long-term dependency makes the complex memory of an LSTM redundant. In urban environments, the static features of the scene (where the road goes) are often more influential than the specific nuances of the temporal sequence.
Limitations and Future Work
Despite the impressive speed, the authors note a few areas for improvement. The current model is deterministic, predicting a single "mean" trajectory; future work should incorporate multi-modality to account for multiple possible paths at junctions. Additionally, the model currently lacks social awareness, as it does not explicitly model how vehicles react to each other. Finally, the authors suggest adding specific loss functions or "human knowledge" constraints to prevent "off-road" errors where predicted paths clip the curb.
Summary updated