Multi-sensor Fusion SLAM from the Nadir View for UAV Localization and Mapping

Yawen Li, George Vosselman, Francesco NexISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2025 · OpenReview

Abstract

A single sensor on unmanned aerial vehicles (UAVs) cannot provide stable and accurate trajectory prediction in outdoor low-altitude environments. Moreover, most UAV datasets primarily focus on the low-altitude forward-facing view, with limited coverage of the nadir view. To solve this problem, this study presents a multi-sensor fusion SLAM for UAV localization and mapping from the nadir view in real-time. This method integrates monocular images, IMU measurements, and GNSS coordinates, combining the advantages of each sensor to achieve accurate and reliable state estimation. First, the sensors are initialized and aligned to ensure a consistent reference frame. Subsequently, tracking and local mapping are conducted to establish the system’s midterm stability. Finally, the optimization function is formulated using a factor graph that integrates visual, inertial, GNSS, keyframe proximity, and designed yaw factors. The system is evaluated using the MARS dataset, and the experimental results demonstrate improved drift reduction and enhanced positioning accuracy.

Introduction

While Simultaneous Localization and Mapping (SLAM) is a mature field, most Unmanned Aerial Vehicle (UAV) research focuses on forward-facing cameras. However, many commercial and scientific missions—such as aerial surveying and agricultural monitoring—rely on nadir (downward-looking) views.

Nadir-view SLAM faces unique hurdles. Monocular cameras suffer from scale ambiguity, and Visual-Inertial systems eventually drift without a global reference. Furthermore, when a drone flies in a straight line looking straight down, the "yaw" (its heading) can become mathematically unobservable, leading to significant orientation errors. Most existing datasets and algorithms are not optimized for these specific flight patterns.

Methodology

The proposed system is built upon a tightly-coupled Mono-Inertial-GNSS SLAM framework. Unlike standard visual-inertial tracking, this approach utilizes a factor graph to fuse all sensor data into a single optimization problem.

System architecture of the multi-sensor fusion SLAM framework, highlighting the factor graph optimization components.

The optimization function integrates five distinct factors:

  1. Visual Factor: Standard reprojection error from camera features.
  2. Inertial Factor: IMU pre-integration to handle high-frequency motion.
  3. GNSS Factor: Provides a global anchor to eliminate long-term position drift.
  4. Keyframe Proximity Factor: Unlike standard SLAM, which mostly looks at frames adjacent in time, this factor looks for frames that are physically near each other in 3D space. This is crucial for drones flying parallel strips.
  5. Designed Yaw Factor: To solve the unobservable heading problem, the authors calculate a heading based on GNSS velocity vectors and use it as a soft constraint to keep the drone’s orientation stable.

Experimental Evaluation

The researchers evaluated their work using the MARS dataset, a benchmark specifically designed for multi-sensor UAV fusion. They focused on the HKairport_GNSS02 and HKisland_GNSS02 sequences, collected via a DJI M300 RTK quadrotor equipped with a Hikvision RGB camera, an IMU, and a ZED F9P GNSS receiver.

Feature extraction and matching from the nadir view in (a) airport and (b) coastal environments.

Key Results

The authors compared their full multi-factor method against a baseline version of ORB-SLAM3 modified to include GNSS:

  • Accuracy: In the Airport sequence, the Absolute Trajectory Error (ATE) dropped from 1.32m to 0.89m (a 32.68% improvement). In the Island sequence, it dropped from 1.02m to 0.75m (a 26.07% improvement).
  • Drift Reduction: The percentage of drift relative to the total path length was reduced to as low as 0.041%.
  • Real-time Performance: The system demonstrated near-real-time capability, processing flight data at speeds comparable to the flight duration.

Conclusion

By prioritizing spatial proximity and introducing a designed yaw factor, this framework effectively addresses the rotational drift and scale ambiguity inherent in nadir-view UAV flights. Tight coupling of GNSS measurements in the backend optimization allows for real-time correction of IMU bias and visual scale. Future work will focus on integrating RTK-based GNSS directly into the fusion process to reach centimeter-level precision and testing in high-canopy or dense urban environments.

Summary updated

Abstract

Articles | Volume X-2/W2-2025 ArticleMetricsRelated articles Articles | Volume X-2/W2-2025 https://doi.org/10.5194/isprs-annals-X-2-W2-2025-109-2025 © Author(s) 2025. This work is distributed under the Creative Commons Attribution 4.0 License. https://doi.org/10.5194/isprs-annals-X-2-W2-2025-109-2025 © Author(s) 2025. This work is distributed under the Creative Commons Attribution 4.0 License. Articles | Volume X-2/W2-2025 ArticleMetricsRelated articles 29 Oct 2025 | 29 Oct 2025 Multi-sensor Fusion SLAM from the Nadir View for UAV Localization and Mapping Yawen Li, George Vosselman, and Francesco Nex Yawen Li https://orcid.org/0000-0001-8822-7485 × Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands George Vosselman https://orcid.org/0000-0001-8813-8028 × Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands Francesco Nex https://orcid.org/0000-0002-5712-6902 × Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands Keywords: UAV, Multi-sensor fusion SLAM, Nadir view, State estimation, Factor graph Abstract. A single sensor on unmanned aerial vehicles (UAVs) cannot provide stable and accurate trajectory prediction in outdoor low-altitude environments. Moreover, most UAV datasets primarily focus on the low-altitude forward-facing view, with limited coverage of the nadir view. To solve this problem, this study presents a multi-sensor fusion SLAM for UAV localization and mapping from the nadir view in real-time. This method integrates monocular images, IMU measurements, and GNSS coordinates, combining the advantages of each sensor to achieve accurate and reliable state estimation. First, the sensors are initialized and aligned to ensure a consistent reference frame. Subsequently, tracking and local mapping are conducted to establish the system’s midterm stability. Finally, the optimization function is formulated using a factor graph that integrates visual factor, inertial factor, GNSS factor, keyframe proximity factor, and designed yaw factor. The system is evaluated using the MARS dataset, and the experimental results demonstrate improved drift reduction and enhanced positioning accuracy. Download & links Article (PDF, 9338 KB) Download & links Article (9338 KB) Metadata XML BibTeX EndNote Share document.addEventListener("DOMContentLoaded", function () { const mobileShareElement = document.querySelector(".mobile-native-share"); if (navigator.share) { // Native sharing is available if (mobileShareElement) { mobileShareElement.style.display = "block"; } } else { // Native sharing is NOT available if (mobileShareElement) { mobileShareElement.style.display = "none"; } } }); How to cite. Li, Y., Vosselman, G., and Nex, F.: Multi-sensor Fusion SLAM from the Nadir View for UAV Localization and Mapping, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., X-2/W2-2025, 109–116, https://doi.org/10.5194/isprs-annals-X-2-W2-2025-109-2025, 2025.
BibTeX
@article{li2025multi,
  author = {Yawen Li and George Vosselman and Francesco Nex},
  title = {{Multi-sensor Fusion SLAM from the Nadir View for UAV Localization and Mapping}},
  journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  year = {2025},
  url = {https://openreview.net/forum?id=5Vy9uvEj2a},
}
Figure