TL;DR
GoToNet is a lightweight, monocular navigation framework that enables drones to explore unknown environments in real-time using only a single forward-looking RGB camera. By predicting "Goto" and "Lookat" pixels, the system optimizes flight paths to maximize scene exposure, significantly outperforming existing methods in both exploration efficiency and computational speed.
What Problem Does This Paper Address?
Autonomous exploration in unknown, GPS-denied, or communication-denied environments (like indoor disaster zones or underground mines) remains a massive hurdle for robotics. Traditional Simultaneous Localization and Mapping (SLAM) methods are often too computationally heavy for Micro Aerial Vehicles (MAVs) and typically require expensive, heavy depth sensors like LiDAR.
Existing "Next-Best-View" (NBV) algorithms, which decide where a robot should look next to gather the most information, often rely on discrete action spaces (e.g., "turn 25 degrees left") and voxel-based maps that are slow to process. The authors of GoToNet sought a "one-look" approach: a system that can take a single RGB image and instantly decide the best tactical move to uncover the most "unseen" territory.
Methods
The GoToNet framework operates in a three-step deep learning pipeline:
- Depth Prediction: An RGB image is fed into a DenseDepth network (a UNet-like model with a DenseNet-161 encoder) to generate a depth map.
- Gradient Extraction: The system applies a Sobel-Feldman operator to the depth map. This highlights sharp changes in depth, which are strong indicators of open spaces or hidden areas behind objects.
- Navigation Instruction: A modified VGG16 network processes the gradient map to predict two specific sets of coordinates:
- Goto Pixel: Defines the 3D direction the agent should move.
- Lookat Pixel: Defines where the camera should point in the next step to maximize exposure.
The "secret sauce" lies in how they generated the training labels. In a simulator, they "painted" seen areas of a 3D scene. They then simulated a 3x3 grid of possible next poses, counted how many "unseen" pixels each would reveal, and used a weighted average of these results to determine the optimal Goto and Lookat coordinates.
Datasets
The researchers developed a novel dataset generator using the Microsoft AirSim simulator (built on Unreal Engine).
- 3D Exploration Dataset: 48,400 data samples consisting of RGB images, corresponding depth maps, and optimized Goto/Lookat pixel coordinates.
- 2D Exploration Dataset: 37,172 samples for a simplified horizontal-plane movement scenario.
- Environment: A custom-made indoor scene featuring multiple rooms, diverse objects, and complex pathways to test navigation logic.
Key Insights and Ideas
- Gradients over Raw Depth: The authors found that training on gradient maps is more effective than raw depth. Gradients explicitly signal "edges" and potential openings, preventing the model from overfitting on specific depth values and helping it identify "holes" in the scene that need exploring.
- Continuous Action Space: Unlike many models that choose from a few fixed directions, GoToNet predicts pixel coordinates. This allows for much finer, more accurate movement and rotation adjustments.
- The "Non-Exposing" Fail-Safe: If the drone faces a wall and no direction reveals new voxels (a "non-exposing case"), the model is trained to head toward the deepest point in the current depth map to find an exit.
- Constant-Time Decision Making: Because the model only requires a single look to make a decision, its compute time doesn't grow as the map gets larger, making it ideal for long-duration missions.
Key Results
GoToNet was compared against "Learn-to-Score," a state-of-the-art voxel-based NBV method. The results were striking:
- Exploration Efficiency: GoToNet exposed 850% more new voxels per camera pose than the baseline.
- Target Acquisition: In tests aimed at finding a random target, GoToNet brought the drone 10.5% closer to the goal.
- Speed: The mean step time on an RTX 2080 Max-Q was just 5.67 seconds, compared to 31.58 seconds for the baseline—a nearly 6x speedup.
- Surface Coverage: GoToNet saw 8.89% of the total surface voxels in a trial, while the baseline only managed 0.89%.
Limitations and Future Work
The primary limitation acknowledged is the Sim-to-Real gap. While the AirSim environment is high-fidelity, transferring these models to physical drones involves dealing with real-world lighting, motion blur, and hardware constraints.
Future directions suggested by the authors include:
- Cooperative Swarms: Developing multi-agent capabilities where drones share "seen" data to divide and conquer a scene.
- End-to-End Optimization: Further shrinking the model size to reduce runtime even more.
- Dense Sampling: Increasing the 3x3 grid used during training data generation to 9x9 or higher to potentially improve the "optimality" of the predicted paths.
Summary updated