Real-time processing of ultra-high-resolution UAV imagery is critical for time-sensitive applications such as disaster response and infrastructure monitoring. However, the computational demands of feature extraction and bundle adjustment (BA) for 50MP+ images often necessitate downsampling, which sacrifices essential spatial detail. This paper introduces a novel real-time BA framework that operates on full-resolution imagery by employing adaptive patch-based feature tracking. By integrating onboard GNSS/IMU data with coarse global digital surface models (DSMs), the system predicts patch movement and ensures spatial consistency across flight strips. Validation on 50MP datasets demonstrates that the proposed method achieves precise camera orientation in under two seconds per image pair on standard hardware without GPU acceleration.
Introduction
In the wake of natural disasters or during critical infrastructure inspections, the immediate availability of georeferenced 3D data is paramount. Traditional photogrammetric workflows, while accurate, are often too slow for live flight operations. Standard Bundle Adjustment—the refinement of camera poses and 3D points—becomes a significant bottleneck when processing ultra-high-resolution imagery. Most existing solutions compromise by downsampling images or using incremental methods that are prone to drift and fail to align overlapping flight strips. This research proposes a localized BA framework designed to provide global-level accuracy at real-time speeds.
Proposed Methodology
The framework is built upon a localized approach to bundle adjustment that optimizes processing efficiency through three core innovations:
1. Adaptive Patch-Based Tracking
Rather than performing feature extraction across an entire 50MP frame, the algorithm divides the image into a user-defined grid (typically 150x150 pixel patches). Feature extraction and tracking are restricted to these small windows, drastically reducing the search space and computational load.
Illustration of the adaptive patch-based feature tracking mechanism.
To maintain patch consistency across frames, the system leverages the UAV’s navigation data (GNSS/IMU) and a coarse DSM (such as SRTM or TanDEM-X). By projecting the image footprint onto the terrain, the system accurately predicts patch displacement in subsequent frames, even in complex topography. This "geometry-first" approach narrows the feature matching problem from a global search to a highly localized verification.
3. Cluster-Based Optimization
The system utilizes "sliding clusters" of images (e.g., 12-frame windows) for optimization. Unlike standard SLAM systems that only consider sequential frames, this method identifies and incorporates overlapping images from adjacent flight strips via footprint intersection. This ensures inter-strip consistency and prevents the parallax distortions common in long-track incremental pipelines.
Overview of the proposed real-time bundle adjustment workflow.
The framework was evaluated using a real-world dataset from the 2023 Türkiye earthquake, consisting of 60 images (50MP each) captured in two flight strips by the DLR Modular Aerial Camera System (MACS). Testing was conducted on a standard office laptop (Intel Core i7, 32GB RAM) to simulate edge-computing constraints.
Metric
Traditional Incremental BA
Proposed Method
Total Processing Time
962.69 s
66.45 s
Feature Matching Time
530.31 s
0.62 s
Mean Reprojection Error
0.727 px
0.710 px
Runtime per Image Pair
~16.0 s
1.11 s
The results indicate a staggering reduction in matching time and a total speedup of over 14x. Qualitatively, the proposed method eliminated the "ghosting" artifacts and drift observed in traditional methods when transitioning between flight strips, maintaining high-fidelity mapping across the entire survey area.
Conclusion and Future Work
This research demonstrates that full-resolution UAV imagery can be processed in real-time by substituting exhaustive searches with geometry-guided patch tracking. Future developments will focus on automating hyperparameter tuning (such as patch size) based on flight altitude and resolution. Additionally, the authors plan to implement a fallback mechanism for environments where DSM data is unavailable and move toward direct onboard deployment on UAV hardware.
@article{iz2025real,
author = {Selim Ahmet Iz and Francesco Nex and Norman Kerle and Henry Meissner and Ralf Berger},
title = {{Real-Time Bundle Adjustment for Ultra-High-Resolution UAV Imagery Using Adaptive Patch-Based Feature Tracking}},
journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
year = {2025},
url = {https://openreview.net/forum?id=M6O1zog4rj},
}