Abstract
In view of the urgent need for high-resolution and high-precision infrared image data for geological environmental monitoring, we have developed a lightweight optical butting infrared imaging system for unmanned aerial vehicles (UAVs) to obtain infrared image data. In this paper, we proposed a robust mosaicking method combination of as-projective-as-possible warps and similarity transformation, which uses multiple images to generate a wide viewing field panorama. It is developed based on an analysis of projective transformation and similarity transformation. The proposed method simultaneously takes into consideration the flexibility of alignment and the preservation of image shapes. On the overlapping regions, it performs as-projective-as-possible warps. On the non-overlapping regions, it gradually becomes a similarity warp. The results show that the proposed method weakens the “ghosting,” shape distortion, and area distortion.
Introduction
Creating wide-field infrared maps for geological monitoring—such as identifying landslides or groundwater seepage—requires stitching hundreds of individual drone-captured images. However, traditional mosaicking techniques often fail due to two primary hurdles:
- Parallax and "Ghosting": Because UAVs move and terrain is rarely perfectly flat, overlapping images do not align perfectly using global homography, leading to double-images or "ghosts."
- Shape Distortion: Advanced algorithms designed to fix ghosting often over-compensate, causing the edges of the final panorama to look unnaturally stretched or warped, known as the "funhouse mirror" effect.
Furthermore, lightweight infrared cameras typically suffer from significant lens distortion and "trapezoidal distortion" caused by the drone’s tilted flight angle during acquisition.
Methodology
The proposed system utilizes a three-stage geometric correction and mosaicking pipeline to ensure high-precision results.
1. Non-linear Distortion Correction
The process begins with removing radial lens distortion using a "division model." This step ensures that straight lines in the physical world are correctly represented as straight lines in the digital image, providing a clean baseline for further processing.
2. Trapezoidal Correction
By leveraging real-time data from an onboard Inertial Navigation System (INS/GNSS), the system calculates the drone's exact attitude (roll, pitch, and yaw). It then projects the tilted images onto a flat North-East-Down (NED) coordinate system, effectively creating a "top-down" orthophoto view.
3. The Hybrid Warp (Core Innovation)
The core of the mosaicking method is a transition between mathematical models based on image overlap:
- Overlapping Regions: The system employs As-Projective-As-Possible (APAP) warps. This provides the local flexibility required to align features perfectly, even in the presence of parallax.
- Non-overlapping Regions: To prevent the stretching common in projective warps, the algorithm gradually transitions into a Similarity Transformation. This preserves the original aspect ratio and shapes of objects at the edges of the panorama.
Experimental Setup
The study utilized a custom-built experimental system designed for lightweight airborne operations.
- Hardware: A DJI Matrice 600 UAV equipped with a DJI RONIN-MX pan-tilt gimbal, two lightweight infrared cameras, and an SBG Ellipse2-N INS/GNSS sensor.
- Data Sequences: Three distinct infrared image sequences were evaluated, featuring resolutions of 960×1280 and 447×1277. The test site included topographic fluctuations with elevation differences of approximately 6 meters, providing a challenging environment for traditional stitching.
Key Results and Discussion
The hybrid approach successfully balanced alignment flexibility with shape preservation. Compared to industry standards like AutoStitch and Microsoft ICE, the proposed method eliminated "ghosting" in complex terrain where other software failed.
While more computationally intensive than global homography, the method remains efficient. Processing a sequence took approximately 16.3 seconds, only a slight increase over the 14.4 seconds required for standard APAP, while delivering significantly higher visual quality. The system demonstrated robustness in generating large-scale panoramic mosaics even under significant terrain fluctuations.
Conclusion
This research presents a robust pipeline for UAV-based infrared imaging that addresses the critical issues of parallax and edge distortion. By combining hardware-informed pre-processing with a hybrid APAP-similarity warp, the system produces high-resolution panoramas suitable for precise geological monitoring. Future work will focus on optimizing computational speed to move toward real-time processing and testing the algorithm in diverse fields such as disaster rescue and agricultural mapping.
Summary updated