TL;DR
This paper presents a redundant perception and state estimation architecture for autonomous race cars, designed to maintain reliability even if a critical sensor fails. By independently estimating track boundaries using both LiDAR and stereo cameras, and fusing this data with a slip-aware, failure-detecting velocity estimator, the system enables safe, high-speed navigation (up to 90 km/h and 1.7G lateral acceleration) without the need for duplicated hardware.
What Problem Does This Paper Address?
In autonomous racing, vehicles operate at the absolute limits of their handling capabilities. At these extremes, a single sensor failure—such as a camera being blinded by the sun or a GPS losing signal—can result in a catastrophic crash. Traditionally, redundancy is achieved by simply doubling the hardware (e.g., installing two LiDARs), which drastically increases cost, weight, and computational load.
The authors tackle the challenge of achieving algorithmic redundancy. They aim to build a system that can extract the exact same semantic and spatial information (specifically, the color and position of track-delimiting cones) from entirely different, complementary sensor modalities. Furthermore, they address the challenge of state estimation in racing, where high wheel slip makes standard wheel odometry highly inaccurate.
Methods
The authors developed independent perception pipelines for LiDAR and cameras, tied together by a robust state estimator and SLAM backend:
- LiDAR Pipeline: After motion distortion compensation and ground removal, the system clusters points to find cones. To determine cone color (blue vs. yellow) without a camera, they use a Convolutional Neural Network (CNN) that analyzes the distinct intensity gradients of the LiDAR returns caused by the cones' reflective bands.
- Camera Pipeline: To save compute, YOLOv2 is used to detect cones in only the left image of a stereo pair. The system then uses epipolar geometry to spatially propagate the bounding boxes to the right image. A neural network extracts keypoints, SIFT features are matched, and the points are triangulated for a 3D position estimate.
- Slip-Aware Velocity Estimation: An Extended Kalman Filter (EKF) fuses data from an IMU, Ground Speed Sensor (GSS), GNSS, and Wheel Speed Sensors (WSS). Crucially, the process model incorporates a Pacejka tire model to explicitly account for wheel slip. It also features a probabilistic failure detection module (using Chi-square and variance tests) to isolate and ignore drifting or dead sensors.
- SLAM: A fastSLAM 2.0 particle filter asynchronously fuses the independent LiDAR and camera observations, updating a probabilistic map of landmark positions and colors.
Datasets
The authors utilized self-acquired datasets gathered using the gotthard driverless race car (built by the AMZ team at ETH Zürich). Data was collected across five different testing locations and during the Formula Student Germany competitions. Ground truth for mapping was obtained using a highly accurate Leica Totalstation, and ground truth for velocity was captured via an optical ground speed sensor.
Key Insights and Ideas
- LiDAR intensity is a proxy for color: You don't strictly need a camera to tell a blue cone from a yellow one. Because the cones have standardized reflective bands, a CNN can learn the distinct vertical intensity patterns in the LiDAR point cloud to classify color, proving more robust than rule-based heuristics.
- Smart stereo matching saves compute: Running heavy object detection models on both images of a stereo pair is computationally wasteful. Detecting on one image and projecting the region of interest to the other using known camera geometry is a highly efficient shortcut for real-time robotics.
- Slip-awareness is non-negotiable in racing: At high speeds, race cars experience up to 20% longitudinal wheel slip. Standard kinematic models fail here; incorporating dynamic tire force models into the EKF is essential to keep the velocity estimate from drifting.
- Algorithmic redundancy works: By feeding independent perception streams into a probabilistic SLAM framework, the car can survive the total failure of either the camera or the LiDAR and still successfully map and navigate the track.
Key Results
- LiDAR Color Accuracy: The CNN achieved ~96% classification accuracy for cones within 5 meters and generalized significantly better than rule-based approaches when faced with novel cone types (e.g., cones with different competition stickers).
- Velocity Estimation: The slip-aware EKF achieved a Root Mean Square Error (RMSE) of just 0.14 m/s compared to ground truth. When integrated over a 310m track, the positional drift was less than 1.5m (< 0.5% error).
- SLAM Accuracy: The system achieved a map RMSE of 0.2 meters using only LiDAR, and 0.25 meters using only stereo cameras—both accurate enough to safely finish a race in the event of a single-modality failure.
- Real-World Performance: The full stack was successfully deployed on the gotthard driverless car, enabling it to autonomously navigate unknown tracks at speeds up to 90 km/h.
Limitations and Future Work
The authors note a few physical limitations of their perception pipelines. The LiDAR color estimation is capped at a distance of 5 meters because point clouds become too sparse beyond that range to resolve the intensity patterns. Similarly, the stereo camera depth estimation is capped at 10 meters because disparity (and thus depth accuracy) drops off significantly at larger distances.
For future work, the authors suggest improving the sensor failure detection module by incorporating post-state analysis to better catch subtle, complex sensor degradation that current statistical tests might miss.
Summary updated