Compact RGBD Surface Models Based on Sparse Coding

Michael Ruhnke, Liefeng Bo, Dieter Fox, Wolfram Burgard

Introduction

Robots navigating the real world require detailed 3D maps containing both geometry and texture (RGB-D data). However, raw point clouds are memory-intensive, and standard compression techniques like Octomaps often force a trade-off between large file sizes and significant discretization errors.

This paper identifies a key opportunity: human-made environments are repetitive. Walls, floors, and furniture share similar geometric primitives and textures. By moving beyond rigid "bag-of-words" models toward a flexible representation based on sparse coding, the authors propose a system that is as efficient as a library-based approach but as accurate as a point cloud.

Methodology: Sparse Coding for 3D Surfaces

The proposed pipeline treats 3D surfaces as signals to be compressed via sparse coding.

Point Cloud Decomposition

Raw data is decomposed into small, overlapping 3D surface patches (e.g., 20cm x 20cm). Each patch is oriented based on its surface normal to ensure consistency across different viewpoints.

Dictionary Learning with weighted K-SVD (wK-SVD)

The core innovation is the learning of a "dictionary"—a set of basis patches—from the data. Real-world scans often contain holes due to occlusions. To address this, the authors extended the standard K-SVD algorithm into weighted K-SVD (wK-SVD). This uses a binary mask to ignore undefined areas during training, focusing the dictionary learning on observed surfaces.

Decomposition of a surface patch into its constituent components for sparse encoding.

Sparse Encoding and Storage

Instead of storing every point, the system stores a "sparse code"—a linear combination of dictionary entries. The final model consists only of the dictionary, the 3D positions/orientations of the patches, and their associated sparse codes.

Experimental Results

The researchers evaluated their approach using several datasets, including a large-scale RGBD Corridor dataset consisting of 707 registered Kinect scans.

Compression and Accuracy

For the RGBD Corridor dataset, 3.35 GB of raw data was compressed into a 10.2 MB file—a reduction of over 300x. Compared to Octomap (a robotics standard), the proposed method produced a file 4x smaller while achieving smoother surface reconstructions and better accuracy. On the Scene IV-A dataset, the method achieved a Root Mean Square Error (RMSE) of 1.6 cm, significantly outperforming the 5.8 cm error of previous patch-based methods.

Reconstruction of an office corridor using the proposed sparse coding method, achieving massive compression while maintaining visual fidelity.

Efficiency

Dictionary learning for a standard dataset takes between 2.5 and 8 minutes on a standard desktop, making the approach practical for offline robotic mapping tasks.

Key Insights: Codes as Features

A significant finding is that these compact codes are also highly descriptive features. Because similar shapes result in similar sparse codes, object detection (e.g., identifying soda cans) can be performed directly in the compressed code space. This is significantly faster than searching through raw 3D data.

Conclusion and Future Work

While the results are promising, dictionary learning remains a computational bottleneck for real-time applications. Future work includes integrating this representation directly into SLAM (Simultaneous Localization and Mapping) processes to allow robots to map larger environments with minimal memory overhead and applying more sophisticated machine learning classifiers to the sparse codes for robust object recognition.

Summary updated

Abstract

In this paper, we describe a novel approach to construct compact colored 3D environment models representing local surface attributes via sparse coding. Our method decomposes a set of colored point clouds into local surface patches and encodes them based on an overcomplete dictionary. Instead of storing the entire point cloud, we store a dictionary, surface patch positions, and a sparse code description of the depth and RGB attributes for every patch. The dictionary is learned in an unsupervised way from surface patches sampled from indoor maps. We show that better dictionaries can be learned by extending the K-SVD method with a binary weighting scheme that ignores undefined surface cells. Through experimental evaluation on real world laser and RGBD datasets we demonstrate that our method produces compact and accurate models. Furthermore, we clearly outperform an existing state of the art method in terms of compactness, accuracy, and computation time. Additionally, we demonstrate that our sparse code descriptions can be utilized for other important tasks such as object detection.
BibTeX
@inproceedings{ruhnke2013compact,
  author = {Michael Ruhnke and Liefeng Bo and Dieter Fox and Wolfram Burgard},
  title = {{Compact RGBD Surface Models Based on Sparse Coding}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2013},
  doi = {10.1609/aaai.v27i1.8549},
}
Figure