TL;DR
This paper introduces an end-to-end deep learning model for autonomous driving specifically designed for temporary road scenarios, which are challenging for traditional systems. By leveraging a Global Context Vision Transformer (GCViT) for feature extraction and a Transformer-based fusion module, the model effectively predicts local paths and controls vehicle movement, demonstrating promising results in both simulation and real-world tests.
What Problem Does This Paper Address?
Autonomous driving has seen significant advancements on structured roads, but temporary road environments—often created by construction or traffic control (e.g., marked by cones)—remain a significant challenge. These scenarios are less predictable and can cause conventional modular autonomous driving systems to fail due to inter-module coupling and the need for complex rule design. The authors aim to address this gap by developing a robust, end-to-end model that directly maps visual input to driving actions, enhancing safety and stability in these dynamic and unstructured environments.
Methods
The proposed model is an End-to-End system with several key components:
- Sensor Data Input Module: Utilizes three RGB cameras (front, left, right) to capture a wide field of view.
- Image Feature Extraction Module: Employs the 'xtiny' version of the Global Context Vision Transformer (GCViT) network. GCViT is chosen for its ability to model both long-range and short-range spatial interactions, enhancing feature extraction from images.
- Transformer Fusion Module: Integrates image features using a Transformer encoder. During training, a Transformer decoder also predicts a local obstacle map (Bird's Eye View probability map), which aids in training robustness. During deployment, only the encoder is used for efficiency.
- Local Path Prediction Module: Predicts local paths (a sequence of waypoints) using a combination of a Transformer encoder's output and a Gated Recurrent Unit (GRU) network. This module predicts paths rather than direct control signals, making it more adaptable.
- Controller Module: A separate Proportion Integration Differentiation (PID) controller takes the predicted local path and computes the vehicle's angular velocity for steering control, ensuring path tracking at a constant speed.
Datasets
- CARLA Simulator: Used to create virtual testing environments.
- Urban Simulation Environments: Used for pre-training the model.
- Customized Temporary Road Map: A specific map within CARLA was created, featuring temporary roads marked by traffic cones, with straight sections, left, and right turns. This was used for fine-tuning and evaluation.
- Experimental Dataset Collection: Data was captured using a rule-based expert model in CARLA and also through manual driving on a physical platform. This dataset includes RGB images from three cameras and target paths. Data augmentation was used to increase diversity.
Key Insights and Ideas
- End-to-End for Temporary Roads: The core idea is that an end-to-end approach can simplify the system and improve robustness in complex, unstructured environments like temporary roads, where modular systems struggle.
- GCViT for Enhanced Feature Extraction: Using GCViT as the backbone for image feature extraction proved superior to traditional CNNs like ResNet, showing faster convergence and better performance, even with a smaller model size.
- Obstacle Map as Training Aid: Including the prediction of a local obstacle map during the training phase significantly reduced the path prediction loss (Lwp), indicating that this auxiliary task improves the model's overall robustness and performance.
- Path Prediction vs. Direct Control: Predicting local paths instead of directly outputting steering angles offers greater interpretability and adaptability to different vehicle hardware.
Key Results
- Simulation Performance: The model achieved an average trajectory deviation of 0.689 m in simulation tests, meeting real-time processing requirements with a path planning time of approximately 80 ms per frame.
- Field Test Performance: In field experiments, the model demonstrated consistent path tracking, with the autonomous driving trajectory closely overlapping the manual driving trajectory. The path planning time increased slightly to around 100 ms due to hardware limitations of the embedded platform.
- Backbone Comparison: GCViT outperformed ResNet18, ResNet26, and ResNet50 in terms of loss convergence and performance for image feature extraction.
Limitations and Future Work
- PID Controller Tuning: The authors note that tuning the three coupled parameters of the PID controller is challenging, and fixed-parameter PID controllers struggle to adapt to dynamically changing systems (e.g., varying vehicle load).
- Future Work:
- Enhance self-driving performance by training with larger datasets in more complex environments.
- Explore the use of Model Predictive Control (MPC) algorithms to improve control performance, potentially overcoming the limitations of PID controllers.
Summary updated