A UAV Image Stitching Method for Complex Urban Environments

Wenyue Niu, Dongwei Qiu, Runze Wu, Zhaowei Wang, Yida Shi2025 · DOI

TL;DR

This paper introduces a novel UAV image stitching method for complex urban environments that improves feature point distribution and stitching accuracy. By combining an optimized ORB algorithm with Gaussian scale-space and dynamic grid division, the method achieves real-time performance on low-power hardware while significantly reducing stitching distortions.

What Problem Does This Paper Address?

Capturing high-resolution aerial imagery with Unmanned Aerial Vehicles (UAVs) is invaluable for applications like environmental monitoring and disaster assessment. However, stitching these images into a seamless panorama, especially in complex urban settings, presents significant challenges. These include:

  • Environmental Interference: Dense buildings, varying heights, intricate street layouts, dynamic occlusions (moving vehicles, pedestrians), and changing lighting conditions can all degrade image quality and make feature matching difficult.
  • Hardware Limitations: UAVs often have limited computational power, making it challenging to process large amounts of image data in real-time using computationally intensive algorithms.
  • Feature Point Distribution: Traditional methods often struggle with uneven distribution of detected feature points. They tend to be overly concentrated in textured areas (like building facades) and sparse in low-texture regions (like sky or smooth surfaces), leading to local alignment errors and overall stitching inaccuracies.
  • Deep Learning Drawbacks: While deep learning methods show promise, they require vast amounts of labeled data, are computationally expensive for training and inference, and can be "black boxes" lacking interpretability, making them less suitable for resource-constrained UAVs.

This paper aims to bridge this gap by developing a lightweight, efficient, and robust image stitching solution specifically for complex urban environments.

Methods

The proposed method is a multi-stage approach that enhances the traditional ORB algorithm:

  1. Image Preprocessing:
    • Locally Adaptive Noise Filtering: Instead of a uniform Gaussian filter, this method dynamically adjusts the filter's standard deviation based on the noise variance within local image patches. This helps to smooth noisy areas without excessively blurring details in cleaner regions.
  2. Bilateral Filtering: Applied after adaptive Gaussian filtering, this technique further refines image quality by smoothing while preserving important edges and details, crucial for feature detection.
  3. Improved ORB Algorithm:
    • Gaussian Scale-Space Pyramid: To capture features at various scales (from building contours to window details), the algorithm constructs a multi-scale representation of the image. FAST corner detection is then applied at each scale.
  4. Dynamic Grid Partitioning: To address uneven feature point distribution, the image is divided into a grid. The size of these grids is dynamically adjusted based on local information entropy. High-entropy (textured) areas get smaller grids to capture more points, while low-entropy areas get larger grids to avoid redundancy. This ensures a more uniform distribution of feature points.
  5. Adaptive Threshold Control: If a grid has too few feature points, the detection threshold is lowered; if it has too many, points with lower responses are discarded.
  6. Global Geometric Optimization:
    • Energy Function: A global energy function is defined to jointly optimize the homography matrix (which describes the geometric transformation between images). This function includes terms for reprojection error (ensuring accurate local matches) and a smoothness constraint (penalizing abrupt changes in the homography to prevent stitching gaps and distortions).
  7. Levenberg-Marquardt Algorithm: This iterative optimization algorithm is used to minimize the energy function, leading to a globally consistent and distortion-free stitched image.

Datasets

The authors created a UAV aerial image dataset specifically for evaluating their method in complex urban environments. This dataset includes scenes with:

  • High-rise dense areas (significant height differences, occlusions).
  • Dynamic occlusion scenarios (moving vehicles and pedestrians).
  • Lighting variation scenes (strong midday light and backlighting).

This dataset appears to be a newly introduced benchmark for testing robustness in challenging urban conditions.

Key Insights and Ideas

  • Balancing Act: The core innovation lies in effectively balancing computational efficiency (inherent to ORB) with improved accuracy and robustness in complex environments. This is achieved by optimizing feature detection and distribution rather than relying on computationally heavy alternatives.
  • Dynamic Adaptation: The dynamic grid partitioning based on local information entropy is a clever way to adapt feature point sampling to the varying textures of urban scenes, directly tackling a major weakness of standard ORB.
  • Global Consistency is Key: The global optimization step, using an energy function with smoothness constraints, is crucial for eliminating the visual artifacts (gaps, distortions) that often plague stitched images, especially those with significant geometric variations.
  • Real-time on a Budget: The method's ability to achieve real-time performance on low-power UAV hardware is a significant practical advantage, making it directly applicable to current drone operations.

Key Results

  • Feature Point Distribution Uniformity: The proposed method achieved a uniformity index of 0.89, a significant improvement of 43.5% over the standard ORB algorithm.
  • Stitching Accuracy (RMSE): The method achieved an RMSE of 3.5 pixels, outperforming traditional ORB (6.2 pixels) and SIFT (4.5 pixels) and approaching the accuracy of deep learning methods.
  • Processing Speed: The algorithm runs at a real-time speed of 45ms per frame, significantly faster than SIFT (120ms) and deep learning methods (300ms), making it suitable for low-performance UAV platforms.
  • Robustness: The method demonstrated strong performance in challenging scenes, including high-rise dense areas (RMSE 3.8 pixels), dynamic occlusion (RMSE 4.2 pixels), and lighting variations.

Limitations and Future Work

  • Limitations: While the method is robust, the authors acknowledge that further improvements in handling extreme environmental conditions like fog or very low light could be beneficial. Parameter sensitivity (e.g., grid size, feature point limits) might still require some manual tuning for optimal performance in highly specific scenarios, though the dynamic nature of the grid helps mitigate this.
  • Future Work: The authors plan to explore:
  • Lightweight Deep Learning Integration: Combining the strengths of deep learning with their optimized feature-based approach.
  • Edge Computing Acceleration: Optimizing the algorithm for deployment on embedded UAV platforms using techniques like model quantization.
  • Temporal Consistency: Incorporating constraints for video streams to reduce accumulated errors in long-term sequences.
  • Multimodal Data Fusion: Integrating data from other sensors (e.g., infrared, LiDAR) to enhance performance in adverse conditions.

Summary updated

Abstract

To address the issues of uneven feature point distribution, environmental interference, and insufficient real-time performance in UAV image stitching in complex urban environments, this paper proposes an improved ORB algorithm based on Gaussian scale-space optimization and dynamic grid division, combined with a global geometric consistency optimization strategy. First, local adaptive noise filtering and bilateral filtering are applied to enhance image quality. Then, multi-scale feature detection is achieved using a Gaussian scale-space pyramid, and dynamic grid division is employed to balance feature point distribution. Finally, a global energy function, including reprojection error and smoothness constraints, is constructed to iteratively optimize the homography matrix and suppress stitching distortions. Experimental results show that the proposed method achieves high processing speed on low-performance hardware platforms, improves feature point distribution uniformity to 0.89, and achieves stitching accuracy (RMSE) of 3.5 pixels, significantly outperforming ORB and SIFT algorithms, while remaining robust in dynamic occlusion and lighting variation scenarios. This method provides a lightweight and efficient solution for UAV image stitching in urban environments, supporting applications such as urban planning and disaster assessment. Future work will explore lightweight deep learning integration and edge computing acceleration to further improve dynamic scene adaptability.
BibTeX
@inproceedings{niu2025uav,
  author = {Wenyue Niu and Dongwei Qiu and Runze Wu and Zhaowei Wang and Yida Shi},
  title = {{A UAV Image Stitching Method for Complex Urban Environments}},
  booktitle = {˜The},
  year = {2025},
  doi = {10.5194/isprs-archives-xlviii-g-2025-1123-2025},
}
Figure