TL;DR
NARUTO is the first neural active reconstruction system capable of operating with unrestricted 6-Degree-of-Freedom (6DoF) movement in large-scale indoor environments. By combining a fast hybrid neural representation with real-time, explicit uncertainty learning, the system autonomously plans paths toward poorly reconstructed areas, boosting environment reconstruction completeness from 73% to over 90%.
What Problem Does This Paper Address?
Active reconstruction is the challenge of commanding a robotic agent or camera to autonomously explore an unknown environment to build a high-fidelity 3D model. While implicit neural representations like Neural Radiance Fields (NeRFs) have revolutionized 3D reconstruction, their heavy computational demands have severely restricted their use in active vision.
Prior neural active mapping systems were forced to limit the agent's movement—constraining them to discrete "teleportation" locations, simple 2D planes, or small object-centric hemispheres. NARUTO breaks through these barriers, addressing the need for a real-time system that can handle continuous 6DoF exploration in large, complex, multi-room indoor spaces.
Methods
NARUTO integrates mapping, uncertainty quantification, and path planning into a unified, real-time loop:
- Neural 3D Mapping: The system uses Co-SLAM as its backbone, which utilizes a multi-resolution hash-grid combined with coordinate encoding. This hybrid approach offers rapid convergence and excellent hole-filling capabilities.
- Explicit Uncertainty Learning: Instead of using an implicit Multi-Layer Perceptron (MLP) to guess uncertainty, NARUTO introduces a learnable Uncertainty Grid ($V_\sigma$). By tying uncertainty directly to depth rendering losses during bundle adjustment, the system dynamically maps out which parts of the 3D space are highly uncertain.
- Uncertainty-Aware Planning: The system aggregates the top-$k$ most uncertain points within the agent's sensing range to identify a target goal. It then utilizes an optimized path planner called Efficient RRT (E-RRT), which speeds up traditional 3D Rapidly-exploring Random Trees by actively attempting direct line-of-sight connections to the goal.
- Active Ray Sampling: During mapping optimization, rather than sampling pixels entirely at random, NARUTO uses its learned uncertainty map to actively sample rays from regions with poor reconstruction quality, acting as a plug-and-play booster for map stability.
Datasets
The authors evaluated NARUTO inside the Habitat Simulator using two photorealistic indoor 3D datasets:
- Replica: A dataset of high-quality, dense digital replicas of indoor spaces (used primarily for ablation studies).
- Matterport3D (MP3D): A large-scale dataset featuring complex, multi-room real-world indoor environments, presenting significant exploration challenges.
Key Insights and Ideas
- Explicit Grids Beat Implicit Networks for Uncertainty: The authors discovered that using an MLP network to predict uncertainty introduces instability. Because an MLP's weights change globally during optimization, updates in one room can unpredictably alter the uncertainty values of unobserved rooms. An explicit 3D uncertainty volume ensures that updates remain strictly local.
- Reachability Filtering is Essential: When aggregating uncertainty to find the next best view, the system uses its E-RRT planner to check if a high-uncertainty zone is actually reachable. If a zone is trapped behind a wall or outside the navigable boundaries, it is filtered out, preventing the agent from getting stuck.
Key Results
- State-of-the-Art Completeness: On the challenging MP3D dataset, NARUTO achieved a 90.18% completion ratio, drastically outperforming the previous state-of-the-art active mapping method (ANM), which sat at 73.15%.
- High-Fidelity Geometry: NARUTO reduced the Mean Absolute Distance (MAD) error to just 1.44 cm compared to ANM's 4.29 cm, demonstrating much tighter alignment with ground-truth surfaces.
- Real-Time Planning Speeds: Thanks to parallel GPU computing and direct-line shortcuts, the E-RRT path planner computes trajectories in just 5.77 milliseconds, compared to the agonizing 19 seconds required by traditional 3D RRT implementations.
Limitations and Future Work
While NARUTO marks a major step forward, a few limitations remain open for future research:
- Assumption of Perfect Localization: The system currently assumes localization (tracking) is perfectly known and focuses entirely on the reconstruction aspect. In real-world robotics, noisy odometry must be handled jointly (Active SLAM).
- Simplified Agent Physics: The agent is modeled as a simple, free-flying sphere. Real robots have non-holonomic motion constraints (e.g., wheeled robots can't instantly move sideways or fly).
- Single-Resolution Uncertainty: The uncertainty grid uses a fixed voxel size (10cm). Transitioning to a multi-resolution uncertainty representation could help capture finer geometric details more efficiently.
Summary updated