Abstract
Drone navigation requires a comprehensive understanding of both visual and geometric information in the 3D world. In this paper, we present VGF-Net (Visual-Geometric Fusion Network), a deep network for the fusion analysis of visual/geometric data and the construction of 2.5D height maps for simultaneous drone navigation in novel environments. Given an initial rough height map and a sequence of RGB images, VGF-Net extracts visual information alongside a sparse set of 3D keypoints that capture geometric relationships. Driven by the data, VGF-Net adaptively fuses this information into a unified Visual-Geometric Representation. This representation is processed by a new Directional Attention Model (DAM), which enhances object relationships and propagates data to dynamically refine the height map and keypoints. This end-to-end system demonstrates remarkable robustness and high accuracy across complex indoor and large-scale outdoor scenes.
Introduction
Autonomous drone navigation in unknown or partially known environments remains a significant challenge. Traditional approaches often face a trade-off between computationally lightweight visual-only methods, which may lack global spatial awareness, and dense 3D mapping techniques, which are often too resource-intensive for real-time deployment on mobile hardware. Furthermore, drones frequently operate with imperfect or outdated spatial data, such as low-resolution satellite imagery.
To address these issues, VGF-Net utilizes 2.5D height maps as a "sweet spot" for spatial representation. These maps provide sufficient geometric detail for obstacle avoidance and path planning while remaining computationally efficient. VGF-Net enables a drone to take a "rough" initial map and use its onboard cameras to refine it in real-time during flight.
Methodology: The VGF-Net Architecture
VGF-Net is an end-to-end architecture designed for visual-geometric fusion. The workflow is divided into three primary stages:
1. Residual Update Strategy
Rather than generating a map from scratch, the network predicts a "residual" — a set of corrections to an existing rough height map. Convolutional layers extract features from both the incoming RGB image sequence and the current height map to identify discrepancies and required updates.
2. Visual-Geometric Representation
The system integrates a standard SLAM (Simultaneous Localization and Mapping) module to identify a sparse set of 3D keypoints. These keypoints are projected onto the 2.5D height map. For each keypoint, the network calculates directional distances to the nearest object boundaries, creating a rich geometric context.
3. Directional Attention Model (DAM)
The core innovation of VGF-Net is the Directional Attention Model. The DAM propagates information from sparse keypoints to the entire height map. By "attending" to relevant geometric information along specific directions, the model refines object boundaries and updates the estimated positions of keypoints dynamically.
Experiments and Results
The researchers evaluated VGF-Net using a new 3D Urban Navigation Dataset comprising seven large-scale city models (e.g., New York, Chicago, Shenzhen) and the S3DIS dataset for indoor environments.
Robustness and Accuracy
To test the model's ability to "fix" errors, the authors intentionally introduced disturbances—such as translating objects or changing heights—into the ground-truth maps. VGF-Net demonstrated remarkable robustness, significantly outperforming baseline fusion methods even when initial maps were 80% dissimilar to the ground truth.
Navigation Performance
In outdoor benchmarks, VGF-Net achieved a navigation success rate of 82-85% without dedicated depth sensors, surpassing state-of-the-art methods reliant on estimated depth maps. In indoor S3DIS tests, the success rate reached 92% with depth data and 76% using only RGB and height maps.
Real-time Feasibility
Efficiency is critical for drone deployment. The mapping module operates in approximately 0.2 seconds on a mobile GPU, confirming its suitability for real-time onboard processing.
Conclusion
VGF-Net provides a robust solution for simultaneous navigation and mapping by effectively fusing visual cues with sparse geometric data. By leveraging 2.5D height maps and the novel Directional Attention Model, the system enables drones to navigate complex, novel environments with high accuracy and efficiency. Future work will explore more controllable visual representations and the integration of uncertainty mapping to further enhance autonomous exploration.
Summary updated