BEVCast: Path-Adaptive BEV for Efficient End-to-End Autonomous Driving

Qiren Wang, Mingyu Guo, Zixuan Wang, Weisong ShiarXiv 2026 · arXiv

TL;DR

BEVCast introduces a path-adaptive Bird’s-Eye-View (BEV) representation that concentrates high-resolution computation only where the vehicle is predicted to drive. By using a two-stage approach—a coarse pass to identify the future path and a fine pass to refine those specific areas—the model achieves better planning accuracy and safety with only 28% of the queries and 66% of the encoder time compared to standard uniform-grid models.

What Problem Does This Paper Address?

Most modern end-to-end autonomous driving models rely on a dense BEV grid. Whether it's BEVFormer, UniAD, or VAD, these models typically spend the same amount of computation on every cell in a large square (e.g., 100m x 100m) around the vehicle.

However, this is fundamentally inefficient. In a typical 3-second planning horizon, a vehicle might only cross 5 out of 625 cells in a coarse grid. The vast majority of high-resolution computation is "wasted" on areas the car will never reach. While some prior works try to solve this by focusing only on objects or candidate trajectories, they often discard the surrounding scene context entirely, which is risky for general situational awareness. BEVCast asks: Can we make the resolution follow the task instead of a fixed grid?

Methods

BEVCast employs a two-stage pipeline to create a "mixed-resolution" BEV:

  1. Coarse Pass & Heat Head: The model first encodes the entire scene at a coarse resolution (4.096m cells). A lightweight "heat head" (an MLP) predicts which of these coarse cells the vehicle is likely to drive through. This head is trained using the vehicle's actual recorded path from logs—a form of "free" supervision.
  2. Refinement & Fine Pass: Based on a fixed "refinement budget" (e.g., 10%), the model selects the top-$k$ highest-heat cells and subdivides them into fine-grained 1.024m cells.
  3. Mixed-Resolution Encoding: The main encoder processes a query set consisting of these new fine cells plus the remaining unrefined coarse cells. This ensures the model has high precision for the path it needs to navigate while maintaining a coarse "peripheral vision" of the rest of the scene.
  4. Trajectory Regression: Finally, a transformer decoder reads this mixed-resolution BEV to output a single predicted trajectory. Notably, the model operates without HD maps or explicit navigation commands.
Figure 1. Overview of BEVCast at the 10% refinement budget. The coarse pass encodes 625 cells of 4.096 m over the whole scene and its heat head predicts where the ego vehicle will drive. The 62 cells with the highest heat are subdivided into 1.024 m cells and the fine pass encodes these 992 fine cells together with the 563 unrefined coarse cells, from which the trajectory head regresses one trajectory. Both encoders read the shared image features and the ego motion embedding and the coarse memory of the previous keyframe. The full grid baseline encodes 10,000 cells of 1.024 m in one pass with the same encoder architecture.

Datasets

The authors evaluated BEVCast on two major benchmarks:

  • nuScenes: The industry standard for BEV perception and planning, using the open-loop protocol.
  • NAVSIM: A newer, planning-centric benchmark that uses the Extended Predictive Driver Model Score (EPDMS) to better capture human-like driving quality.

Key Insights and Ideas

  • Context is King, but Precision is Local: You don't need to see a sidewalk 50 meters away in millimeter detail to avoid hitting it, but you do need high precision for the lane boundaries you are currently navigating. BEVCast proves that keeping the background coarse is better than dropping it entirely.
  • Free Supervision via Logs: By using the actual path driven in the training data as the ground truth for the "heat head," the model learns where to spend its compute budget without requiring any manual labeling.
  • Path-Adaptive > Proposal-Centric: Unlike models that only look at pre-defined candidate paths, BEVCast's resolution adapts dynamically to the predicted intent, creating a "corridor" of high resolution that extends into the future.
Figure 2. Learned selection at the 10% refinement budget on three validation frames. The top row shows the camera that faces the recorded path with the recorded future in green and the predictions of the full grid in blue and of BEVCast in orange. The bottom row shows the heat in pale blue on the coarse grid over the whole 102.4 m range with the 62 refined cells outlined in orange and subdivided into 1.024 m cells and the annotated agents in grey.

Key Results

BEVCast demonstrates that efficiency and accuracy are not a zero-sum game:

  • Efficiency: At a 10% refinement budget, it uses only 28% of the BEV queries and reduces encoder time to 66% compared to a full high-resolution grid.
  • Accuracy (nuScenes): It achieved a mean displacement error of 1.078m, outperforming the full-grid baseline of 1.149m.
  • Safety: The drivable area violation rate dropped significantly from 2.29% to 1.46%, suggesting that the focused resolution helps the model better respect road boundaries.
  • NAVSIM Performance: It reached an EPDMS of 82.51 (vs. 81.28 for the baseline), placing it competitively among published planners despite not using navigation commands.

Limitations and Future Work

While impressive, the authors note several areas for improvement:

  • Single-Seed Training: Results are based on models trained once with a single seed; more robust statistical averaging could further validate the small differences between different refinement budgets.
  • Open-Loop Constraints: Like many current papers, evaluation is primarily open-loop. Testing how this adaptive resolution performs in a closed-loop simulator (where the model's actions affect the future state) is a logical next step.
  • Navigation Commands: The current model doesn't use navigation commands (like "turn left at the next intersection"). Integrating these could help resolve ambiguities in complex junctions where multiple paths are plausible.

Summary updated

Abstract

Camera-based driving models build a dense bird's-eye-view (BEV) grid and spend the same computation on every cell of it, although the vehicle crosses only a few of those cells within the planning horizon. We present BEVCast, a two-stage BEV whose resolution adapts to the predicted driving path. A coarse pass encodes the whole scene on a grid of 4.096 m cells. A heat head trained on the recorded path assigns a heat to every coarse cell and the cells with the highest heat are refined into 1.024 m cells under a fixed budget. The remaining cells keep their coarse resolution, so the trajectory head reads a mixed-resolution BEV that still covers the entire scene and needs neither an HD map nor a navigation command. Against a full grid of 1.024 m cells with the same architecture and training recipe, BEVCast at a 10% refinement budget encodes 28% of the BEV queries and needs 66% of the BEV encoder time at batch size 1. On nuScenes its mean displacement error is 1.078 m against 1.149 m and its drivable area violation rate is 1.46% against 2.29%. On NAVSIM navtest its mean displacement error is 0.577 m against 0.598 m and its extended predictive driver model score is 82.51 against 81.28. Next to published planners BEVCast reaches a comparable error on both benchmarks without a navigation command. The resolution of a driving BEV can therefore follow the task instead of the grid.
BibTeX
@misc{wang2026bevcast,
  author = {Qiren Wang and Mingyu Guo and Zixuan Wang and Weisong Shi},
  title = {{BEVCast: Path-Adaptive BEV for Efficient End-to-End Autonomous Driving}},
  year = {2026},
  eprint = {2611.08631},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.08631},
}
Figure