Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Driving

Ziying Song, Caiyan Jia, Lin Liu, Hongyu Pan, Yongchang Zhang, Junming Wang, Xingyu Zhang, Shaoqing Xu, Lei Yang, Yadan Luo

TL;DR

MomAD (Momentum-Aware Driving) addresses the issue of "vehicle trembling" and unstable control in end-to-end autonomous driving by introducing trajectory and perception momentum from historical frames. By aligning multi-modal trajectory proposals with prior paths and mixing historical planning queries, MomAD achieves smoother, safer, and more temporally consistent driving, especially during complex maneuvers like turns.

What Problem Does This Paper Address?

Most state-of-the-art end-to-end autonomous driving frameworks operate on a "one-shot" or frame-by-frame basis. They ingest sensor data at time $t$, predict a trajectory, and repeat the process at time $t+1$. While effective for simple scenarios, this approach suffers from severe temporal inconsistency.

Because the model selects the highest-scoring trajectory independently at each frame, slight variations in perception or occlusions can cause the chosen path to jump drastically between timesteps. This results in "vehicle trembling"—undesirable directional shifts and oscillations that make for an uncomfortable and potentially dangerous ride. Furthermore, single-frame perception is highly vulnerable to sudden occlusions, missing critical context that a human driver would naturally retain using memory and situational momentum.

Comparison of planning paradigms: Deterministic, Multi-modal, and Momentum Planning (Ours).

Methods

The authors propose the Momentum-Aware Driving (MomAD) framework to inject temporal stability into end-to-end planners. The architecture consists of two primary components acting on two distinct levels of "momentum":

  1. Topological Trajectory Matching (TTM) [Trajectory Momentum]: Instead of blindly picking the highest-scoring multi-modal trajectory proposal at the current frame, TTM uses the Hausdorff Distance to measure how well candidate trajectories align globally with the path planned in the previous timestep. It selects the candidate that minimizes this distance, enforcing smooth path continuity.
  2. Momentum Planning Interactor (MPI) [Perception Momentum]: To prevent the model from becoming overly biased by the current frame's perception, the MPI uses a Long-horizon Query Mixer. It passes historical planning queries through an LSTM to simulate temporal evolution and cross-attends the current best planning query with these historical queries. This enriches the current query with long-term spatiotemporal context.

Additionally, to combat upstream perception noise, MomAD incorporates a Robust Instance Denoising Module during training, which injects controlled Gaussian noise perturbations into instance features, forcing the planning network to learn to ignore perception artifacts.

To quantitatively evaluate this behavior, the authors also introduce a new metric: Trajectory Prediction Consistency (TPC), which measures the spatial disparity between consecutive trajectory predictions over overlapping time horizons.

Datasets

The framework was validated across three setups:

  • nuScenes: The standard benchmark for open-loop autonomous driving tasks (detection, tracking, mapping, and planning).
  • Turning-nuScenes: A challenging validation subset curated by the authors. Since standard nuScenes is dominated by straight roads (which can mask planning instability), they extracted 17 scenes (680 samples) specifically involving sharp turns, roundabouts, and intersections.
  • Bench2Drive: A closed-loop evaluation protocol operating within the CARLA simulator to test reactive, real-time performance across 44 interactive scenes.

Key Insights and Ideas

  • The Multi-Modal Trap: While multi-modal trajectory planning provides excellent action diversity, selecting the top trajectory based purely on the current frame's maximum score introduces severe temporal jitter. The "best" mode can flip back and forth between frames due to minor perception noise.
  • The Illusion of Straight Roads: Standard open-loop benchmarks like nuScenes hide planning deficiencies because going straight requires minimal correction. Evaluating models on a curated turning dataset exposes how fragile frame-by-frame planners truly are when continuous, smooth steering adjustments are mandatory.
  • Dual-Level Momentum: True driving stability requires both geometric consistency (matching the physical curves of past trajectories via TTM) and semantic consistency (carrying over historical intent and environmental memory via MPI).

Key Results

MomAD achieves state-of-the-art performance across both open-loop and closed-loop benchmarks:

  • Long-Horizon Planning (6-second prediction): On the challenging Turning-nuScenes dataset, MomAD reduced the collision rate by 26% and improved the Trajectory Prediction Consistency (TPC) metric by 0.97m (33.45%) compared to the baseline SparseDrive model.
  • Standard nuScenes: MomAD achieved a SOTA L2 displacement error of 0.60m and a TPC of 0.54m, outperforming UniAD and VAD.
  • Closed-Loop (Bench2Drive): In real-time reactive testing, MomAD improved the driving success rate by up to 16.3% over the multi-modal variant of VAD and 8.4% over SparseDrive, while simultaneously boosting trajectory smoothness (comfort score) by up to 7.2%.

Limitations and Future Work

The authors note that while MomAD significantly stabilizes long-horizon trajectory predictions, it still suffers from a degree of mode collapse. This is primarily induced by the standard teacher-forcing approach used during trajectory regression training, which inherently limits the diversity of generated paths.

To address this, the authors point to future work exploring generative approaches, such as diffusion models and speculative decoding, to safely expand trajectory diversity without sacrificing real-time computational efficiency.

Summary updated

Abstract

End-to-end autonomous driving frameworks enable seamless integration of perception and planning but often rely on one-shot trajectory prediction, which may lead to unstable control and vulnerability to occlusions in single-frame perception. To address this, we propose the Momentum-Aware Driving (MomAD) framework, which introduces trajectory momentum and perception momentum to stabilize and refine trajectory predictions. MomAD comprises two core components: (1) Topological Trajectory Matching (TTM) employs Hausdorff Distance to select the optimal planning query that aligns with prior paths to ensure coherence;(2) Momentum Planning Interactor (MPI) cross-attends the selected planning query with historical queries to expand static and dynamic perception files. This enriched query, in turn, helps regenerate long-horizon trajectory and reduce collision risks. To mitigate noise arising from dynamic environments and detection errors, we introduce robust instance denoising during training, enabling the planning model to focus on critical signals and improve its robustness. We also propose a novel Trajectory Prediction Consistency (TPC) metric to quantitatively assess planning stability. Experiments on the nuScenes dataset demonstrate that MomAD achieves superior long-term consistency (>=3s) compared to SOTA methods. Moreover, evaluations on the curated Turning-nuScenes shows that MomAD reduces the collision rate by 26% and improves TPC by 0.97m (33.45%) over a 6s prediction horizon, while closedloop on Bench2Drive demonstrates an up to 16.3% improvement in success rate.
BibTeX
@inproceedings{song2025don,
  author = {Ziying Song and Caiyan Jia and Lin Liu and Hongyu Pan and Yongchang Zhang and Junming Wang and Xingyu Zhang and Shaoqing Xu and Lei Yang and Yadan Luo},
  title = {{Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Driving}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02089},
}
Figure