Decomposition of Neural Discrete Representations for Large-Scale 3D Mapping

Minseong Park, Suhan Woo, Euntai Kim

TL;DR

DNMap (Decomposition-based Neural Mapping) introduces a novel way to compress 3D maps by decomposing complex neural features into a small set of shared "component vectors." By learning to compose these vectors rather than indexing a massive dictionary, the method achieves high-fidelity large-scale mapping with up to 97% less feature storage and significantly faster training times than previous discrete representation methods.

What Problem Does This Paper Address?

In the world of autonomous robots and self-driving cars, building accurate 3D maps of large-scale environments is essential. Modern approaches often use Implicit Neural Representations, where a neural network learns a Signed Distance Function (SDF) to represent surfaces. To scale these to city-sized environments, researchers use "feature volumes"—grids or octrees that store local data (embeddings) to help the network understand specific areas.

The catch? These feature volumes are memory hogs. Continuous embeddings take up massive amounts of storage. While "discrete" representations (like Vector Quantization) can compress this data by using a codebook (a dictionary of features), they introduce a new bottleneck: training becomes incredibly slow and memory-intensive because the system has to search the entire dictionary for every single point. DNMap tackles this "indexing bottleneck" to make large-scale neural mapping both storage-efficient and practical to train.

Methods

The core innovation is the Decomposition Strategy. Instead of trying to pick one feature from a giant list of $2^B$ possibilities (indexing), DNMap learns a small set of $B$ shared "component vectors."

  1. Decomposition & Composition: Any local feature is represented as a combination of these shared component vectors. Think of it like building with a small set of LEGO bricks rather than picking a pre-molded plastic toy from a warehouse.
  2. Binary Indicators: Each voxel in an octree stores a simple binary vector. These bits act as "switches" that determine which component vectors are added together to create the final embedding.
  3. Straight-Through Estimator (STE): Since binary switches aren't naturally differentiable (you can't take a gradient of a "0" or "1"), the authors use an STE approach to allow the model to learn these discrete choices via standard backpropagation.
  4. Hybrid Representation: To ensure the map doesn't lose global context, they add a tiny layer of low-resolution continuous embeddings at the top level of the octree.
  5. Efficient Implementation: During training, they interpolate the binary indicators before composing the features, which avoids redundant calculations and speeds up the process significantly.
Fig. 1: The comparison between indexing [27] and our proposed decomposition strat- egy. Indexing requires exploring the entire embedding space (2B) to optimize discrete representations. In contrast, our approach only needs to learn B indicators.

Datasets

The authors evaluated DNMap on three major benchmarks:

  • MaiCity: A synthetic, noise-free LiDAR dataset of urban roads, used to test ideal-case reconstruction.
  • Newer College: A challenging real-world dataset collected with a handheld LiDAR at Oxford University, containing significant sensor noise and motion distortion.
  • KITTI (Odometry Sequence 00): Used to demonstrate the model's ability to perform incremental mapping (building the map bit-by-bit as the sensor moves).

Key Insights and Ideas

  • Composition over Indexing: The "Aha!" moment here is that large-scale environments are repetitive. You don't need a unique feature for every corner; you just need a few "basis" patterns that can be combined in different ways. This shift from searching a dictionary to "building from blocks" is what enables the massive speed and memory gains.
  • Discrete as a Regularizer: Interestingly, the discrete nature of the representation seems to make the model more robust to noise. In the Newer College dataset, DNMap produced smoother, more accurate meshes than continuous methods, likely because the discrete "bottleneck" prevents the model from over-fitting to sensor noise.
  • Linearity is a Shortcut: By realizing that their composition process is linear, the authors moved the heavy computation from the voxel level to the query level, ensuring that training speed doesn't tank as the map grows.
Fig. 2: The pipeline of our DNMap. When a query point x is given, the voxels to which the query point belongs are determined at each level of the octree O. For the determined discrete embedding voxels at each level, the voxel corner embeddings are composed from the component vector set E according to the composition indicators stored in the corners. The query embeddings at each level z0, ···, zL−1zc are determined via trilinear interpolation of the voxel corner embeddings. Finally, the query embedding z is determined as the sum of all interpolated features, and the decoder fΘ takes the query embedding as input and outputs a signed distance value.

Key Results

  • Storage Efficiency: On the MaiCity dataset, DNMap (8-bit) reconstructed dense meshes using only 8.5% of the feature storage required by the baseline SHINE-Mapping. On the Newer College dataset, this dropped even further to just 3%.
  • Training Speed: DNMap trained approximately 1.4x faster than previous state-of-the-art discrete methods (VQ-AD) because it avoids the computational cost of exploring the entire embedding space.
  • Mapping Quality: Despite the heavy compression, DNMap maintained or exceeded the F-score (a measure of reconstruction accuracy) of continuous methods. On Newer College, it achieved an F-score of 93.60 compared to the baseline's 92.96.

Limitations and Future Work

The authors note that while DNMap is excellent at capturing repetitive structural patterns, it still struggles with very thin surfaces. This is a common "weak point" for implicit neural representations. To fix this, one might need to reduce voxel sizes or increase octree depth, but that would trade off some of the hard-won memory efficiency.

Future work will likely focus on finding a way to preserve these tiny local details without ballooning the memory footprint, perhaps by specializing certain component vectors for high-frequency details.

Summary updated

Abstract

Learning efficient representations of local features is a key challenge in feature volume-based 3D neural mapping, especially in large-scale environments. In this paper, we introduce Decomposition-based Neural Mapping (DNMap), a storage-efficient large-scale 3D mapping method that employs a discrete representation based on a decomposition strategy. This decomposition strategy aims to efficiently capture repetitive and representative patterns of shapes by decomposing each discrete embedding into component vectors that are shared across the embedding space. Our DNMap optimizes a set of component vectors, rather than entire discrete embeddings, and learns composition rather than indexing the discrete embeddings. Furthermore, to complement the mapping quality, we additionally learn low-resolution continuous embeddings that require tiny storage space. By combining these representations with a shallow neural network and an efficient octree-based feature volume, our DNMap successfully approximates signed distance functions and compresses the feature volume while preserving mapping quality. Our source code is available at https://github.com/minseong-p/dnmap.
BibTeX
@inproceedings{park2024decomposition,
  author = {Minseong Park and Suhan Woo and Euntai Kim},
  title = {{Decomposition of Neural Discrete Representations for Large-Scale 3D Mapping}},
  booktitle = {European Conference on Computer Vision},
  year = {2024},
  doi = {10.1007/978-3-031-73220-1\_21},
}
Figure