BETAV: A Unified BEV-Transformer and Bézier Optimization Framework for Jointly Optimized End-to-End Autonomous Driving

Rui Zhao, Ziguo Chen, Yuze Fan, Fei Gao, Yuzhuo MenSensors 2025 · DOI

TL;DR

This paper introduces BETAV, an end-to-end autonomous driving framework that unifies Bird's Eye View (BEV) Transformer-based perception with Bézier curve-based trajectory planning. BETAV tackles challenges in 3D perception accuracy and trajectory smoothness, achieving state-of-the-art performance on the nuScenes dataset and demonstrating robust generalization.

What Problem Does This Paper Address?

Traditional autonomous driving systems often rely on a modular, hierarchical architecture. While effective, these systems can suffer from error accumulation between modules, requiring meticulous interface design and detailed environmental modeling. End-to-end approaches aim to simplify this by directly mapping sensor inputs to driving actions, but they face their own hurdles:

  • Perception Challenges: Accurately understanding complex 3D environments from 2D camera data is difficult, especially with occlusions and dynamic agents. Inferring precise real-world coordinates and agent states is a persistent issue.
  • Planning Challenges: Generating smooth, kinematically feasible trajectories in real-time is demanding. Traditional methods often produce discrete trajectory points, leading to suboptimal smoothness and requiring complex post-processing. Data dependency and generalization to unseen scenarios are also significant concerns.

BETAV aims to bridge these gaps by proposing a unified framework that enhances both perception and planning, leading to more reliable and robust autonomous driving.

Methods

BETAV is built upon two core components:

  1. BEV-Transformer for Perception and Prediction:
    • Multi-view Fusion: The framework takes input from six synchronized RGB cameras, covering a 360° view.
  2. BEV Encoding: Vision Transformers (ViTs) are used to encode these multi-view images into a Bird's Eye View (BEV) representation. This BEV representation captures both spatial and temporal features, providing a comprehensive understanding of the driving scene.
  3. Query-Based Interaction: A novel query-based interaction mechanism is employed. Learnable queries (BEV queries, agent queries, map queries) interact with fused camera features and historical BEV features through self-attention and cross-attention mechanisms. This allows for efficient fusion of multi-modal sensor signals and extraction of explicit environmental features and implicit contextual cues.
  4. Task-Specific Decoders: Decoders then extract high-level semantic information (map, agent, and traffic element information) from the BEV representation.
  5. Bézier Curve-Based Trajectory Planning:
    • Parametric Representation: Instead of outputting discrete points, the planning decoder generates control points for a cubic Bézier curve. This provides a compact, continuous, and parameterized trajectory representation.
  6. Inherent Smoothness and Feasibility: Bézier curves naturally ensure motion smoothness, kinematic feasibility, and computational efficiency.
  7. Constraint-Based Optimization: A set of carefully designed constraints are incorporated to ensure safety and adherence to driving rules. These include:
  8. Minimum Turning Radius Constraint: Ensures the trajectory respects the vehicle's steering limits.
  9. Curvature Continuity Constraint: Prevents excessive curvature and ensures smooth transitions.
  10. Collision Avoidance Constraint: Maintains a safe distance from other agents.
  11. Road Boundary Adherence Constraint: Keeps the trajectory within drivable regions.
  12. Motion Direction Constraint: Aligns the trajectory with the vehicle's and road's direction.
  13. Imitation Learning: The entire framework is trained using imitation learning, minimizing the discrepancy between predicted actions and expert demonstrations.

Datasets

  • nuScenes: The primary dataset used for training and evaluation. It's a large-scale, diverse dataset with 1000 driving scenarios, featuring synchronized multi-camera data, LiDAR, and radar, along with meticulously annotated 3D bounding boxes.
  • Bench2Drive: Used for generalization testing. This benchmark provides curated scenarios to evaluate the framework's adaptability to different driving situations.

Key Insights and Ideas

  • BEV as a Unified Representation: Using BEV as the central representation for perception effectively fuses multi-view camera information and facilitates downstream planning.
  • Query-Based Interaction for Richer Perception: The query-based mechanism allows the model to selectively attend to relevant parts of the scene and historical information, leading to more accurate and interpretable scene understanding.
  • Bézier Curves for Smooth and Efficient Planning: Shifting from discrete trajectory points to Bézier curve control points offers a more elegant and inherently smooth representation, simplifying planning and improving kinematic feasibility.
  • Synergistic Perception-Planning: The framework demonstrates that tightly integrating advanced perception (BEV-Transformer) with a constrained, continuous planning method (Bézier curves) leads to superior overall performance.
  • Importance of Explicit Constraints: The inclusion of specific kinematic and safety constraints directly in the planning loss function is crucial for generating safe and reliable trajectories.

Key Results

  • State-of-the-Art Performance on nuScenes: BETAV achieves superior performance in trajectory prediction and planning.
  • L2 Displacement Error: Achieves an average error of 0.49 m, which is 38.3% lower than the previous state-of-the-art (VAD-Base's 0.72 m).
  • Collision Rate: Reports an average collision rate of 0.16%, a 27.3% reduction compared to VAD-Base (0.22%).
  • Consistent Long-Term Predictions: The framework shows minimal error growth for predictions up to 3 seconds, outperforming methods that exhibit significant error accumulation over time.
  • Robust Generalization: BETAV demonstrates strong performance across diverse and challenging scenarios in the Bench2Drive benchmark, particularly excelling in dynamic interaction scenarios (95.46% route completion).
  • Ablation Studies Highlight Component Importance: Removing specific query mechanisms (map, agent, ego-vehicle) significantly degrades performance, underscoring the value of the multi-modal interaction design. The Bézier parameterization is shown to be crucial for trajectory smoothing and kinematic feasibility.

Limitations and Future Work

  • Computational Footprint: The reliance on transformer architectures for real-time BEV encoding may pose challenges for deployment on resource-constrained platforms.
  • Adaptability to Extreme Maneuvers: While Bézier curves offer smoothness, their continuity assumptions might limit adaptability in highly dynamic environments with abrupt behavioral changes.
  • Severe Occlusion Scenarios: Pure vision-based perception still faces challenges in severe occlusion scenarios, suggesting the need for complementary sensors.

Future Work Directions:

  • Optimize Computational Efficiency: Explore lightweight transformer variants or hybrid architectures (CNNs + attention) to reduce computational load.
  • Enhance Bézier Decoder: Incorporate adaptive curvature constraints and uncertainty-aware planning modules for improved robustness.
  • Integrate Real-World Traffic Rules: Develop modules for explicit traffic rule reasoning and multi-agent interaction modeling.
  • Open-World Validation: Conduct further validation in open-world environments with long-tail edge cases and diverse sensor configurations to bridge the gap between simulation and real-world deployment.

Summary updated

Abstract

End-to-end autonomous driving demands precise perception, robust motion planning, and efficient trajectory generation to navigate complex and dynamic environments. This paper proposes BETAV, a novel framework that addresses the persistent challenges of low 3D perception accuracy and suboptimal trajectory smoothness in autonomous driving systems through unified BEV-Transformer encoding and Bézier-optimized planning. By leveraging Vision Transformers (ViTs), our approach encodes multi-view camera data into a Bird's Eye View (BEV) representation using a transformer architecture, capturing both spatial and temporal features to enhance scene understanding comprehensively. For motion planning, a Bézier curve-based planning decoder is proposed, offering a compact, continuous, and parameterized trajectory representation that inherently ensures motion smoothness, kinematic feasibility, and computational efficiency. Additionally, this paper introduces a set of constraints tailored to address vehicle kinematics, obstacle avoidance, and directional alignment, further enhancing trajectory accuracy and safety. Experimental evaluations on Nuscences benchmark datasets and simulations demonstrate that our framework achieves state-of-the-art performance in trajectory prediction and planning tasks, exhibiting superior robustness and generalization across diverse and challenging Bench2Drive driving scenarios.
BibTeX
@inproceedings{zhao2025betav,
  author = {Rui Zhao and Ziguo Chen and Yuze Fan and Fei Gao and Yuzhuo Men},
  title = {{BETAV: A Unified BEV-Transformer and Bézier Optimization Framework for Jointly Optimized End-to-End Autonomous Driving}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25113336},
}
Figure