TL;DR
This paper introduces an automatic method to calibrate the boresight misalignments of airborne LiDAR and camera systems, enabling direct georeferencing of images without ground control points. By leveraging LiDAR point clouds to create "Virtual Control Points" (VCPs), the method accurately aligns aerial images with 3D spatial data, crucial for applications requiring rapid mapping.
What Problem Does This Paper Address?
Airborne LiDAR systems provide dense 3D point clouds, while integrated digital cameras capture high-resolution aerial imagery. For accurate fusion and analysis, these datasets must be precisely aligned. A key challenge is determining the "boresight misalignments" – the small, often unmeasured rotational offsets between the LiDAR sensor, the camera, and the Inertial Measurement Unit (IMU)/GPS system. Traditional calibration methods require dedicated test fields and ground control points (GCPs), which are expensive, time-consuming, and impractical for rapid response scenarios like disaster relief. This paper aims to solve this by developing an automatic calibration method that uses the LiDAR data itself as a reference, eliminating the need for external GCPs.
Methods
The core of the proposed method is the creation and utilization of Virtual Control Points (VCPs). Here's how it works:
- Tie Point Extraction: The process begins by automatically detecting and matching "tie points" in overlapping aerial images using the Speed-Up Robust Features (SURF) algorithm. SURF is chosen for its balance of accuracy and computational efficiency.
- Space Intersection: Using the initial exterior orientation elements (position and attitude) provided by the GPS/IMU system, object coordinates for these tie points are calculated through space intersection.
- VCP Creation: The crucial step is replacing the calculated elevation of these object points with the elevation directly from the LiDAR point cloud. This creates a VCP, which is essentially a 3D point whose X and Y coordinates are derived from image matching, but whose Z coordinate is taken from the highly accurate LiDAR data. This assumes the LiDAR data represents a relatively flat surface.
- Boresight Misalignment Estimation: The method then iteratively refines the boresight misalignments. It calculates the distance between the original tie points in the image and the projected VCPs. By minimizing this distance through adjustments to the boresight angles (rotations), the system iteratively converges on the correct misalignments.
- Iterative Refinement: This process repeats until the calculated errors stabilize or a maximum number of iterations is reached, yielding the calibrated boresight misalignments.
Datasets
The method was validated using two real-world datasets:
- Dataset 1: Acquired in Xi'an, Shaanxi province, China, using a Leica RCD105 camera and a Leica ALS60 LiDAR system.
- Dataset 2: Acquired in Ningbo, Zhejiang Province, China, using a Leica RCD30 camera and a Leica ALS70 LiDAR system.
Both datasets consist of LiDAR point clouds and corresponding aerial images. The paper does not introduce new benchmark datasets but uses existing data from operational projects.
Key Insights and Ideas
- LiDAR as a Self-Calibration Tool: The most significant insight is the clever use of LiDAR point clouds not just for mapping, but as an intrinsic calibration reference. This bypasses the need for external ground control.
- Virtual Control Points (VCPs): The concept of VCPs is central. By leveraging the high vertical accuracy of LiDAR in flat areas, the method creates reliable control points directly from the fused data, making the calibration process robust.
- Automatic and Efficient: The method is fully automated, from tie point extraction to boresight calibration, significantly reducing manual effort and processing time compared to traditional methods.
- Flat Area Assumption: The VCP concept relies on the assumption of relatively flat terrain or surfaces. This is a practical limitation but is addressed by VCP selection criteria (planarity, slope tolerance).
Key Results
- Accuracy Improvement: After boresight misalignments calibration, the planar accuracy of the georeferenced images showed a significant improvement, with RMSE values decreasing by 2-3 times. For Dataset 1, the RMSEs were approximately 0.39m (X), 0.51m (Y), and 0.64m (XY) after calibration, compared to much higher values before.
- Comparison: The proposed method achieved competitive accuracy compared to other literature methods, particularly those that also rely on LiDAR data. It outperformed methods that used manual tie point selection or required more complex pre-processing steps.
- Stability: The calibration results tend to stabilize when a sufficient number of images (e.g., >16) and strips are used, indicating the robustness of the iterative approach.
- Visual Verification: Mosaicked images showed seamless linear features, and LiDAR point clouds registered well with the georeferenced images, providing strong visual confirmation of the method's effectiveness.
Limitations and Future Work
- Flat Area Dependency: The method's reliance on VCPs derived from flat areas means it might be less effective in highly mountainous or complex terrain without modifications.
- Error Sources: The paper acknowledges that while boresight misalignments are addressed, other potential error sources (e.g., sensor noise, atmospheric effects) are not explicitly included in the current adjustment model.
- Simultaneous Data: The current method assumes LiDAR and imagery are collected simultaneously. The authors suggest future work could explore using pre-collected LiDAR data or point clouds generated from stereo matching, provided a POS system is available.
- Inclusion of Other Errors: Further research could explore extending the adjustment model to incorporate other error sources for even greater accuracy.
Summary updated