SensorX2car: Sensors-to-car calibration for autonomous driving in road scenarios

Guohang Yan, Zhaotong Luo, Zhuochun Liu, Yikang LiarXiv 2023 · DOI · arXiv

Introduction

In the rapidly evolving field of autonomous driving, the precision of a vehicle's perception system is paramount. While much attention is given to "extrinsic calibration"—the spatial relationship between different sensors—a critical but often overlooked step is sensor-to-car calibration. This process aligns the sensors' coordinate systems with the vehicle's own chassis and driving axis. SensorX2car introduces the first open-source toolbox designed to perform this calibration online, during normal driving, without the need for specialized targets or controlled environments.

The Challenge of Sensor-to-Car Calibration

Autonomous vehicles rely on a suite of sensors, including cameras, LiDAR, radar, and GNSS/INS, to navigate. If a sensor, such as a LiDAR unit, is tilted even slightly relative to the car's forward axis, the vehicle's software may misinterpret the trajectory of obstacles or the curvature of the road. Traditionally, this alignment is performed in a factory setting using expensive checkerboard targets. However, real-world conditions—such as road vibrations, thermal expansion, and mechanical wear—can cause these sensors to shift over time.

Comparison between vehicle and sensor coordinate systems showing yaw, pitch, and roll misalignments.

Existing "targetless" calibration methods often depend on specific environmental features, like perfectly straight highways or clearly marked lane lines. SensorX2car aims to overcome these limitations by providing a robust, general-purpose solution that works across diverse road scenarios by leveraging the inherent geometry of the driving environment.

Methodology: A Multi-Modal Approach

The SensorX2car toolbox implements four distinct calibration strategies, each tailored to the unique physical properties of the sensor being calibrated.

Camera-to-Car Calibration

The camera calibration module utilizes a sophisticated deep learning architecture to extract geometric cues from the environment. By identifying the vanishing point (VP) and horizon line (HL) from single images, the system can mathematically derive the camera's roll, pitch, and yaw angles relative to the vehicle.

Deep learning architecture for predicting vanishing points and horizon lines for camera-to-car calibration.

The network employs a Stacked-Hourglass backbone for feature extraction, followed by a Transformer-based encoder-decoder structure. This approach allows the system to function in structured environments even when lane markings are absent, making it significantly more versatile than previous methods that relied solely on road markings.

LiDAR and GNSS/INS Calibration

For LiDAR, the system uses Simultaneous Localization and Mapping (SLAM) to track the vehicle's motion. It employs a RANSAC-based ground plane extraction to determine roll and pitch, while yaw is calculated by aligning the LiDAR-perceived trajectory with the vehicle's actual movement. The GNSS/INS module follows a similar logic, aligning the high-accuracy 6 DoF pose from the navigation system with the vehicle's driving direction.

Radar Calibration

Radar calibration presents a unique challenge due to high noise levels and the sparsity of data. SensorX2car addresses this by analyzing the radial velocity of stationary objects in the environment. By fitting a cosine curve to the relationship between the detected speed of static objects and the vehicle's own velocity, the system can precisely calculate the yaw misalignment. This method relies on the "Trajectory Assumption"—the principle that a vehicle’s orientation is generally consistent with its velocity vector over time. By accumulating data over several minutes of driving, the system filters out outliers caused by sharp turns or dynamic objects.

Experimental Validation and Results

The researchers validated SensorX2car using a combination of the KITTI dataset—manually annotating over 43,000 images with vanishing points—the CARLA simulator for noise-free radar ground truth, and real-world data collected from a test vehicle. The test platform was equipped with high-end sensors, including a Pandar64 LiDAR, Continental ARS 408 Radar, and Novatel GNSS/INS, ensuring a rigorous baseline for comparison.

A key innovation in their evaluation is the use of estimation stability as a performance metric. Since ground truth is difficult to obtain for a moving vehicle, the authors argue that a calibration method is accurate if its results remain consistent across different driving conditions, such as turns and varying slopes.

The results demonstrate impressive precision and consistency:

  • Camera: Rotation angle fluctuations were kept within 0.2° to 0.5° in straight-line sequences.
  • LiDAR: Stability for pitch and yaw reached 0.1°, with height-to-ground (z-axis) estimation accurate to within 6cm.
  • Radar: Despite significant sensor noise, the refined method achieved a yaw stability of approximately 0.5°.
  • Efficiency: Most methods produce a stable calibration result within just a few minutes of normal driving.

Conclusion and Future Work

SensorX2car provides a practical, online solution for maintaining sensor alignment in autonomous vehicles. By focusing on rotation and leveraging environmental cues like vanishing points, it offers a robust alternative to factory-based calibration. While the current version primarily addresses rotation (roll, pitch, and yaw) and assumes fixed translation offsets, it represents a significant step toward self-healing perception systems. Future developments will aim to incorporate full 6 Degrees of Freedom calibration and improve performance during complex, non-linear maneuvers.

The authors have open-sourced the toolbox on GitHub at https://github.com/OpenCalib/SensorX2car.

Summary updated

Abstract

Properly-calibrated sensors are the prerequisite for a dependable autonomous driving system. However, most prior methods focus on extrinsic calibration between sensors, and few focus on the misalignment between the sensors and the vehicle coordinate system. Existing targetless approaches rely on specific prior knowledge, such as driving routes and road features, to handle this misalignment. This work removes these limitations and proposes more general calibration methods for four commonly used sensors: Camera, LiDAR, GNSS/INS, and millimeter-wave Radar. By utilizing sensor-specific patterns: image feature, 3D LiDAR points, GNSS/INS solved pose, and radar speed, we design four corresponding methods to mainly calibrate the rotation from sensor to car during normal driving within minutes, composing a toolbox named SensorX2car. Real-world and simulated experiments demonstrate the practicality of our proposed methods. Meanwhile, the related codes have been open-sourced to benefit the community. To the best of our knowledge, SensorX2car is the first open-source sensor-to-car calibration toolbox. The code is available at https://github.com/OpenCalib/SensorX2car.
BibTeX
@misc{yan2023sensorx,
  author = {Guohang Yan and Zhaotong Luo and Zhuochun Liu and Yikang Li},
  title = {{SensorX2car: Sensors-to-car calibration for autonomous driving in road scenarios}},
  year = {2023},
  eprint = {2301.07279},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.07279},
}
Figure