Real-Time Bundle Adjustment for Ultra-High-Resolution UAV Imagery Using Adaptive Patch-Based Feature Tracking

Selim Ahmet Iz, Francesco Nex, Norman Kerle, Henry Meissner, Ralf Berger

Abstract

Real-time processing of ultra-high-resolution UAV imagery is critical for time-sensitive applications such as disaster response and infrastructure monitoring. However, the computational demands of feature extraction and bundle adjustment (BA) for 50MP+ images often necessitate downsampling, which sacrifices essential spatial detail. This paper introduces a novel real-time BA framework that operates on full-resolution imagery by employing adaptive patch-based feature tracking. By integrating onboard GNSS/IMU data with coarse global digital surface models (DSMs), the system predicts patch movement and ensures spatial consistency across flight strips. Validation on 50MP datasets demonstrates that the proposed method achieves precise camera orientation in under two seconds per image pair on standard hardware without GPU acceleration.

Introduction

In the wake of natural disasters or during critical infrastructure inspections, the immediate availability of georeferenced 3D data is paramount. Traditional photogrammetric workflows, while accurate, are often too slow for live flight operations. Standard Bundle Adjustment—the refinement of camera poses and 3D points—becomes a significant bottleneck when processing ultra-high-resolution imagery. Most existing solutions compromise by downsampling images or using incremental methods that are prone to drift and fail to align overlapping flight strips. This research proposes a localized BA framework designed to provide global-level accuracy at real-time speeds.

Proposed Methodology

The framework is built upon a localized approach to bundle adjustment that optimizes processing efficiency through three core innovations:

1. Adaptive Patch-Based Tracking

Rather than performing feature extraction across an entire 50MP frame, the algorithm divides the image into a user-defined grid (typically 150x150 pixel patches). Feature extraction and tracking are restricted to these small windows, drastically reducing the search space and computational load.

Illustration of the adaptive patch-based feature tracking mechanism.

2. Footprint-Aware Spatial Guidance

To maintain patch consistency across frames, the system leverages the UAV’s navigation data (GNSS/IMU) and a coarse DSM (such as SRTM or TanDEM-X). By projecting the image footprint onto the terrain, the system accurately predicts patch displacement in subsequent frames, even in complex topography. This "geometry-first" approach narrows the feature matching problem from a global search to a highly localized verification.

3. Cluster-Based Optimization

The system utilizes "sliding clusters" of images (e.g., 12-frame windows) for optimization. Unlike standard SLAM systems that only consider sequential frames, this method identifies and incorporates overlapping images from adjacent flight strips via footprint intersection. This ensures inter-strip consistency and prevents the parallax distortions common in long-track incremental pipelines.

Overview of the proposed real-time bundle adjustment workflow.

Experimental Results

The framework was evaluated using a real-world dataset from the 2023 Türkiye earthquake, consisting of 60 images (50MP each) captured in two flight strips by the DLR Modular Aerial Camera System (MACS). Testing was conducted on a standard office laptop (Intel Core i7, 32GB RAM) to simulate edge-computing constraints.

Metric Traditional Incremental BA Proposed Method
Total Processing Time 962.69 s 66.45 s
Feature Matching Time 530.31 s 0.62 s
Mean Reprojection Error 0.727 px 0.710 px
Runtime per Image Pair ~16.0 s 1.11 s

The results indicate a staggering reduction in matching time and a total speedup of over 14x. Qualitatively, the proposed method eliminated the "ghosting" artifacts and drift observed in traditional methods when transitioning between flight strips, maintaining high-fidelity mapping across the entire survey area.

Conclusion and Future Work

This research demonstrates that full-resolution UAV imagery can be processed in real-time by substituting exhaustive searches with geometry-guided patch tracking. Future developments will focus on automating hyperparameter tuning (such as patch size) based on flight altitude and resolution. Additionally, the authors plan to implement a fallback mechanism for environments where DSM data is unavailable and move toward direct onboard deployment on UAV hardware.

Summary updated

Abstract

Articles | Volume X-2/W2-2025 ArticleMetricsRelated articles Articles | Volume X-2/W2-2025 https://doi.org/10.5194/isprs-annals-X-2-W2-2025-73-2025 © Author(s) 2025. This work is distributed under the Creative Commons Attribution 4.0 License. https://doi.org/10.5194/isprs-annals-X-2-W2-2025-73-2025 © Author(s) 2025. This work is distributed under the Creative Commons Attribution 4.0 License. Articles | Volume X-2/W2-2025 ArticleMetricsRelated articles 29 Oct 2025 | 29 Oct 2025 Real-Time Bundle Adjustment for Ultra-High-Resolution UAV Imagery Using Adaptive Patch-Based Feature Tracking Selim Ahmet Iz, Francesco Nex, Norman Kerle, Henry Meissner, and Ralf Berger Selim Ahmet Iz × German Aerospace Center (DLR), Institute of Optical Sensor Systems, Berlin, Germany Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands Francesco Nex https://orcid.org/0000-0002-5712-6902 × Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands Norman Kerle × Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands Henry Meissner × German Aerospace Center (DLR), Institute of Optical Sensor Systems, Berlin, Germany Ralf Berger × German Aerospace Center (DLR), Institute of Optical Sensor Systems, Berlin, Germany Keywords: Real-Time Bundle Adjustment, Local Bundle Adjustment, Rapid Aerial Mapping, High-Resolution Imagery, Direct Georeferencing Abstract. Real-time processing of UAV imagery is crucial for applications requiring urgent geospatial information, such as disaster response, where rapid decision-making and accurate spatial data are essential. However, processing high-resolution imagery in real time presents significant challenges due to the computational demands of feature extraction, matching, and bundle adjustment (BA). Conventional BA methods either downsample images, sacrificing important details, or require extensive processing time, making them unsuitable for time-critical missions. To overcome these limitations, we propose a novel real-time BA framework that operates directly on full-resolution UAV imagery without downsampling. Our lightweight, onboard-compatible approach divides each image into user-defined patches (e.g., N×N grids, default 150×150 pixels) and dynamically tracks them across frames using UAV GNSS/IMU data and a coarse, globally available digital surface model (DSM). This ensures spatial consistency for robust feature extraction and matching between patches. Overlapping relationships between images are determined in real time using UAV navigation system, enabling the rapid selection of relevant neighbouring images for localized BA. By limiting optimization to a sliding cluster of overlapping images, including those from adjacent flight strips, the method achieves real-time performance while preserving the accuracy of global BA. The proposed algorithm is designed for seamless integration into the DLR Modular Aerial Camera System (MACS), supporting large-area mapping in real time for disaster response, infrastructure monitoring, and coastal protection. Validation on MACS datasets with 50MP images demonstrates that the method maintains precise camera orientations and high-fidelity mapping across multiple strips, running full bundle adjustment in under 2 seconds without GPU acceleration. Download & links Article (PDF, 10662 KB) Download & links Article (10662 KB) Metadata XML BibTeX EndNote Share document.addEventListener("DOMContentLoaded", function () { const mobileShareElement = document.querySelector(".mobile-native-share"); if (navigator.share) { // Native sharing is available if (mobileShareElement) { mobileShareElement.style.display = "block"; } } else { // Native sharing is NOT available if (mobileShareElement) { mobileShareElement.style.display = "none"; } } }); How to cite. Iz, S. A., Nex, F., Kerle, N., Meissner, H., and Berger, R.: Real-Time Bundle Adjustment for Ultra-High-Resolution UAV Imagery Using Adaptive Patch-Based Feature Tracking, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., X-2/W2-2025, 73–80, https://doi.org/10.5194/isprs-annals-X-2-W2-2025-73-2025, 2025.
BibTeX
@article{iz2025real,
  author = {Selim Ahmet Iz and Francesco Nex and Norman Kerle and Henry Meissner and Ralf Berger},
  title = {{Real-Time Bundle Adjustment for Ultra-High-Resolution UAV Imagery Using Adaptive Patch-Based Feature Tracking}},
  journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  year = {2025},
  url = {https://openreview.net/forum?id=M6O1zog4rj},
}
Figure