Redundant Perception and State Estimation for Reliable Autonomous Racing

Nikhil Bharadwaj Gosala, Andreas Bühler, Manish Prajapat, Claas Ehmke, Mehak Gupta, Ramya Sivanesan, Abel Gawel, Mark Pfeiffer, Mathias Bürki, Inkyu Sa, Renaud Dubé, Roland Siegwart

Abstract: In autonomous racing, vehicles operate close to the limits of handling and a sensor failure can have critical consequences. To limit the impact of such failures, this paper presents the redundant perception and state estimation approaches developed for an autonomous race car. Redundancy in perception is achieved by estimating the color and position of the track delimiting objects using two sensor modalities independently. Specifically, learning-based approaches are used to generate color and pose estimates, from LiDAR and camera data respectively. The redundant perception inputs are fused by a particle filter based SLAM algorithm that operates in real-time. Velocity is estimated using slip dynamics, with reliability being ensured through a probabilistic failure detection algorithm. The sub-modules are extensively evaluated in real-world racing conditions using the autonomous race car "gotthard driverless", achieving lateral accelerations up to 1.7G and a top speed of 90km/h.

Introduction

In autonomous racing, vehicles operate at the absolute limits of their handling capabilities. At these extremes, a single sensor failure—such as a camera being blinded by the sun or a GPS losing signal—can result in a catastrophic crash. Traditionally, redundancy is achieved by doubling hardware (e.g., installing two LiDARs), which increases cost, weight, and computational load.

This work addresses the challenge of achieving algorithmic redundancy. The goal is to build a system that can extract identical semantic and spatial information—specifically, the color and position of track-delimiting cones—from entirely different, complementary sensor modalities.

Comparison of LiDAR (left) and camera (right) detection modalities for track-delimiting cones.

Redundant Perception Pipelines

LiDAR-Based Color and Pose Estimation

The LiDAR pipeline processes point clouds through motion distortion compensation and ground removal before clustering points to identify cones. A key innovation is the determination of cone color (blue vs. yellow) without camera input. The system utilizes a Convolutional Neural Network (CNN) to analyze vertical intensity gradients in the LiDAR returns. Because cones feature standardized reflective bands, they produce distinct intensity signatures that the CNN can classify with high accuracy.

LiDAR intensity patterns for different cone colors (a) and the CNN architecture used for classification (b).

Camera-Based Perception

To optimize computational efficiency, the camera pipeline uses YOLOv2 to detect cones in only the left image of a stereo pair. Epipolar geometry is then used to spatially propagate these bounding boxes to the right image. Keypoints and SIFT features are matched within these regions to triangulate the 3D position of each cone.

State Estimation and SLAM

Slip-Aware Velocity Estimation

Standard kinematic models often fail in racing environments where vehicles experience up to 20% longitudinal wheel slip. The authors developed an Extended Kalman Filter (EKF) that incorporates a Pacejka tire model to explicitly account for these dynamics. The EKF fuses data from an IMU, Ground Speed Sensor (GSS), GNSS, and Wheel Speed Sensors (WSS). A probabilistic failure detection module (using Chi-square tests) allows the system to isolate and ignore drifting or faulty sensors.

SLAM Backend

A FastSLAM 2.0 particle filter asynchronously fuses the independent LiDAR and camera observations. This updates a probabilistic map of landmark positions and colors, ensuring the vehicle can maintain navigation even if one sensor modality fails completely.

Experimental Results

The system was evaluated using the gotthard driverless race car across multiple testing locations and Formula Student competitions. Key findings include:

  • Perception Accuracy: The LiDAR CNN achieved ~96% color classification accuracy within a 5-meter range.
  • Velocity Reliability: The slip-aware EKF maintained a Root Mean Square Error (RMSE) of just 0.14 m/s. Positional drift over a 310m track was less than 1.5m (< 0.5% error).
  • Mapping Robustness: The SLAM system achieved a map RMSE of 0.2m using only LiDAR and 0.25m using only stereo cameras, proving that either modality is sufficient for safe racing in the event of a failure.
  • High-Speed Performance: The architecture enabled autonomous navigation at speeds up to 90 km/h and lateral accelerations of 1.7G.

Conclusion

By leveraging algorithmic redundancy through independent perception streams and slip-aware state estimation, this architecture provides a robust solution for high-performance autonomous racing. Future work will focus on enhancing sensor failure detection to identify more subtle forms of sensor degradation.

Summary updated

Abstract

In autonomous racing, vehicles operate close to the limits of handling and a sensor failure can have critical consequences. To limit the impact of such failures, this paper presents the redundant perception and state estimation approaches developed for an autonomous race car. Redundancy in perception is achieved by estimating the color and position of the track delimiting objects using two sensor modalities independently. Specifically, learning-based approaches are used to generate color and pose estimates, from LiDAR and camera data respectively. The redundant perception inputs are fused by a particle filter based SLAM algorithm that operates in real-time. Velocity is estimated using slip dynamics, with reliability being ensured through a probabilistic failure detection algorithm. The sub-modules are extensively evaluated in real-world racing conditions using the autonomous race car "gotthard driverless", achieving lateral accelerations up to 1.7G and a top speed of 90km/h.
BibTeX
@inproceedings{gosala2019redundant,
  author = {Nikhil Bharadwaj Gosala and Andreas Bühler and Manish Prajapat and Claas Ehmke and Mehak Gupta and Ramya Sivanesan and Abel Gawel and Mark Pfeiffer and Mathias Bürki and Inkyu Sa and Renaud Dubé and Roland Siegwart},
  title = {{Redundant Perception and State Estimation for Reliable Autonomous Racing}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2019},
  url = {https://openreview.net/forum?id=tmFewTbbet},
}
Figure