Abstract
High-definition maps (HD maps) are a key component of most modern self-driving systems due to their valuable semantic and geometric information. Unfortunately, building HD maps has proven hard to scale due to their cost as well as the requirements they impose in the localization system that has to work everywhere with centimeter-level accuracy. Being able to drive without an HD map would be very beneficial to scale self-driving solutions as well as to increase the failure tolerance of existing ones (e.g., if localization fails or the map is not up-to-date). Towards this goal, we propose an end-to-end approach to mapless driving where the input is raw sensor data and a high-level command (e.g., turn left at the intersection). We then predict intermediate representations in the form of an online map and the current and future state of dynamic agents, and exploit them in a novel neural motion planner to make interpretable decisions taking into account uncertainty. We show that our approach is significantly safer, more comfortable, and can follow commands better than the baselines in challenging long-term closed-loop simulations, as well as when compared to an expert driver in a large-scale real-world dataset.
Introduction: Moving Beyond HD Maps
Most modern self-driving vehicles (SDVs) rely heavily on High-Definition (HD) maps—meticulously detailed digital blueprints containing lane boundaries, traffic rules, and centimeter-level geometry. While effective, HD maps present a significant scaling bottleneck. They are expensive to build, difficult to maintain, and create a hard dependency on perfect localization. If a map becomes outdated or GPS signals falter, the SDV's safety is compromised.
MP3 (Map, Perceive, Predict, and Plan) proposes a shift toward mapless driving. By perceiving the environment in real-time and following high-level routing commands (similar to a human using a standard GPS), MP3 eliminates the need for rigid, pre-computed HD maps.
The MP3 Architecture
MP3 is a unified neural network that transforms raw sensor data into safe driving trajectories through an interpretable pipeline. Unlike "black box" end-to-end models, MP3 produces intermediate spatial representations that allow for human-understandable reasoning.
1. Inputs and Backbone
The model accepts a one-second history of voxelized LiDAR point clouds alongside a high-level goal (e.g., "turn right"). A backbone network extracts deep features from this temporal sensor data.
2. Interpretable Intermediate Representations
MP3 predicts two critical spatial layers:
- Online Map: The model estimates the drivable area, intersections, and the distance/angle to reachable lane centerlines. This effectively builds a local map on the fly.
- Dynamic Occupancy Flow: To handle moving agents, MP3 predicts current occupancy and a "motion field" (velocity vectors). This field describes how occupancy evolves over time, allowing the model to represent multi-modal futures—such as a vehicle that might either go straight or turn.
3. Neural Motion Planning
Rather than regressing steering angles directly, MP3 employs a retrieval-based trajectory sampler. It selects a set of "expert" trajectories from a real-world library, ensuring the proposed paths are physically feasible and human-like. These trajectories are then scored based on the predicted online map and occupancy flow to find the safest, most comfortable path that follows the high-level command.
Key Insights
- Probabilistic Safety: By using dense probabilistic maps instead of discrete object detections, MP3 avoids the "disappearing object" problem common in traditional perception stacks. It accounts for the probability of an object's presence, enhancing safety in occluded or uncertain scenarios.
- Occupancy Flow: Predicting a continuous flow of occupancy naturally handles complex multi-agent interactions and multi-modal behaviors without the need for explicit object tracking or labeling.
Experimental Results
The researchers evaluated MP3 using URBANEXPERT, a large-scale real-world dataset, and tested it in high-fidelity closed-loop simulations. In these challenging environments, MP3 demonstrated superior performance:
- Success Rate: MP3 achieved a 74.39% success rate, vastly outperforming the best baseline (CNMP) at 21.34%.
- Safety and Comfort: The model significantly reduced off-route incidents (14.6% vs. 47% for baselines) and produced smoother trajectories with lower jerk and lateral acceleration.
Conclusion and Future Work
MP3 demonstrates that safe, interpretable mapless driving is achievable by unifying mapping, perception, and prediction into a single planning framework. While the current model still utilizes coarse routing information (e.g., from OpenStreetMap), future iterations aim to integrate visual data for traffic light detection and further distinguish between static and temporarily stopped objects.
Summary updated