Abstract
Creating maps of the world around us is paramount to many applications, including those related to robotics, such as navigation and inspection. Given the computational resource limitations typical of robotic platforms, there is a pressing need for lightweight 3D representations that capture detailed texture and geometric information with minimal storage. Traditional voxel-based approaches require substantial memory resources. On the other hand, neural implicit and 3D Gaussian splatting representations require significant computational power (GPUs) and can hardly run in real time. In this paper, we introduce a novel scene representation, Multi-HexPlanes, that divides 3D environments into large boxes and utilizes the faces of the boxes to encapsulate texture and geometric information. This representation reduces the memory requirement to store the map, making our approach especially suitable for systems with limited memory. Through extensive evaluations on large-scale datasets, we find that our method achieves better performance on rendering and more complete 3D reconstruction. We also demonstrate that our map representation can output dense feature points with rich geometric information for downstream tasks, such as training 3D Gaussian splats. The proposed technique promises substantial improvements in real-time 3D mapping applications, particularly for devices constrained by processing power and storage.
Introduction
Robotic platforms like drones and autonomous inspection bots require high-fidelity 3D maps to navigate and interact with their environments. However, these systems often operate under strict computational and memory constraints. Existing solutions typically fall into two categories: voxel-based maps, which are memory-intensive and often sacrifice resolution for scale, and neural implicit or 3D Gaussian Splatting (3DGS) methods, which offer high visual quality but demand significant GPU power.
Multi-HexPlanes is introduced to fill this gap. It is a novel, lightweight 3D map representation designed for real-time rendering and reconstruction on CPU-only mobile platforms. By shifting from a volumetric representation to a surface-based one, the system achieves a 60% reduction in memory usage compared to traditional voxel-based methods while delivering superior rendering quality.
The Multi-HexPlane Representation
The core innovation of this work is the Multi-HexPlane structure. Instead of filling the interior of a 3D volume with data (which scales cubically), the system divides the environment into large axis-aligned boxes. Each box encapsulates its internal geometry and texture by storing data on its six faces (planes).
To handle the "collision" problem—where multiple surfaces within a box might project to the same spot on a face—the system uses Multi-Channel Pixels. Each pixel in a Multi-HexPlane can store multiple channels, with each channel holding a unique pair of color and distance values. This allows the representation to maintain high resolution and complex geometry without the exponential memory growth seen in voxel grids.
System Pipeline
The Multi-HexPlanes pipeline is designed for incremental, real-time updates. As RGB-D frames are received, the system projects 2D images into 3D point clouds using camera poses and surface normals. These points are then identified within their containing boxes and projected onto the three closest box faces.
The system utilizes a 4-neighbor bilinear interpolation to update stored color, distance, and confidence weights. For visualization, triplanar texturing is used to fetch and blend colors from the relevant faces based on the surface normal, providing a level of visual smoothness typically associated with neural representations but achievable on a standard CPU.
Experimental Results
The authors evaluated Multi-HexPlanes across three datasets: Replica (synthetic accuracy), ScanNet (real-world robustness), and ScanNet++ (high-resolution DSLR for 3DGS initialization).
- Memory Efficiency: Multi-HexPlanes reduced memory consumption by an average of 60% compared to Voxblox at equivalent resolutions.
- Rendering Quality: On the Replica dataset, the method achieved a 30.49 dB PSNR, significantly outperforming Voxblox (28.91 dB) and rivaling some GPU-heavy neural methods like NICE-SLAM.
- Real-Time Performance: The mapping process runs in milliseconds per frame on a standard AMD Ryzen 7 CPU, making it viable for deployment on systems without dedicated GPUs.
- Downstream Utility: The dense feature points extracted from Multi-HexPlanes provide a superior "seed" for 3D Gaussian Splatting compared to traditional Structure-from-Motion (SfM) techniques, leading to faster training and higher final image quality.
Conclusion
Multi-HexPlanes offers a compelling alternative for real-time 3D mapping on resource-constrained devices. By leveraging surface-based dimensionality and multi-channel pixels, it provides high-fidelity reconstruction and rendering with minimal memory overhead. While currently a mapping-only component that relies on external SLAM for tracking, future work aims to integrate these into a unified, lightweight SLAM system.
Summary updated