Abstract
This paper presents VisLanding, a monocular 3D perception-based framework for safe UAV (Unmanned Aerial Vehicle) landing. Addressing the core challenge of autonomous UAV landing in complex and unknown environments, this study innovatively leverages the depth-normal synergy prediction capabilities of the Metric3D V2 model to construct an end-to-end safe landing zones (SLZ) estimation framework. By introducing a safe zone segmentation branch, we transform the landing zone estimation task into a binary semantic segmentation problem. The model is fine-tuned and annotated using the WildUAV dataset from a UAV perspective, while a cross-domain evaluation dataset is constructed to validate the model's robustness. Experimental results demonstrate that VisLanding significantly enhances the accuracy of safe zone identification through a depth-normal joint optimization mechanism, while retaining the zero-shot generalization advantages of Metric3D V2. The proposed method exhibits superior generalization and robustness in cross-domain testing compared to other approaches. Furthermore, it enables the estimation of landing zone area by integrating predicted depth and normal information, providing critical decision-making support for practical applications.
Introduction
Autonomous landing is a critical yet high-risk task for Unmanned Aerial Vehicles (UAVs), especially in unknown or complex terrain where pre-defined markers are unavailable. To land safely, a UAV must identify areas that are flat, obstacle-free, and sufficiently large. Current solutions typically rely on multi-sensor fusion (e.g., LiDAR or RGB-D), which are often too heavy or power-intensive for small UAVs. Purely vision-based methods using monocular cameras offer a lightweight alternative but frequently struggle with "scale ambiguity"—the inability to determine the actual size or distance of objects—and lack geometric reliability.
Standard semantic segmentation models often overfit to specific visual textures, leading to significant performance degradation when deployed in new environments (the "domain gap"). VisLanding addresses these challenges by utilizing 3D geometric priors—specifically depth and surface normals—to guide the identification of safe landing zones.
Methodology
VisLanding is built upon the Metric3D V2 foundation model. Its core innovation is a dual-flow optimization framework that refines geometric and semantic information simultaneously:
- Depth-Normal Flow: This branch inherits the ability to jointly optimize depth and surface normals in a canonical camera space, effectively resolving scale ambiguity.
- Safe Landing Zone (SLZ) Flow: A dedicated branch treats landing zone identification as a binary semantic segmentation problem. Rather than relying solely on RGB data, it performs geometric fusion, using updated depth and normal information as auxiliary features.
- Iterative Refinement: Utilizing a Dense Prediction Transformer (DPT) and ConvGRU modules, the model performs $T$ optimization steps to refine the 3D geometry and the safety mask in tandem.
- Area Estimation: By integrating predicted depth, surface normals, and camera intrinsics, the system projects pixels into 3D space to calculate the actual physical area (in square meters) of candidate sites in real-time.
Experimental Evaluation
The framework was trained and evaluated using two primary datasets: WildUAV (1,329 manually annotated images from a UAV perspective) and the Semantic Drone Dataset (used for cross-domain testing in urban settings).
Key Results
- Superior Generalization: In cross-domain tests, traditional models like SegFormer experienced a 64% drop in performance (mIoU). In contrast, VisLanding’s performance only decreased by 33%, maintaining a significantly higher Mean Intersection over Union of 52.44%.
- Geometric Robustness: By prioritizing geometry (flatness and continuity) over visual texture, VisLanding produces clean, continuous masks even in unfamiliar terrain where baseline models produce fragmented or noisy outputs.
- Real-World Accuracy: Flight tests confirmed that area estimation becomes increasingly accurate as the UAV approaches the target, successfully identifying landing zones of approximately 34m².
Discussion and Future Work
VisLanding demonstrates that 3D geometric priors are more reliable than visual textures for cross-domain UAV tasks. However, the current implementation faces computational challenges. Utilizing a Vision Transformer (ViT) backbone results in a frame rate of less than 5 FPS on edge hardware like the NVIDIA Jetson Orin Nano, which is insufficient for high-speed autonomous flight.
Future research will focus on model distillation and transfer learning to reduce computational overhead, aiming to achieve real-time onboard processing without sacrificing the framework's robust generalization capabilities.
Summary updated