Abstract: RGBD Cameras such as the Microsoft Kinect that can quickly provide usable depth maps have become very affordable, and thus very popular and abundant in recent years. Beyond gaming, RGBD cameras can have numerous applications, including their use in affordable 3D scanners. These cameras however are limited in their ability to capture finer details. We explore the use of additional 3D reconstruction algorithms to enhance the depth maps produced from RGBD cameras, allowing them to capture more detail.
Introduction: The Resolution Gap in Consumer RGBD
Low-cost RGBD (Red, Green, Blue + Depth) cameras have revolutionized robotics and 3D scanning, but they face a significant "resolution" problem. While effective at capturing the general layout of a scene, they struggle with fine surface details. Resulting 3D models often appear "noisy" or "blocky" due to quantization errors and sensor limitations. Existing solutions like KinectFusion address this by averaging data from multiple viewpoints, but this requires static subjects. This paper explores a single-view refinement method to improve depth quality from a single snapshot, bridging the gap between "cheap and noisy" and "high-precision" scanning.
Methodology: Leveraging Infrared Shape from Shading
The researchers build upon the RGBD-Fusion algorithm. Traditionally, Shape from Shading (SfS) estimates 3D geometry by analyzing light reflection in standard color (RGB) images. However, RGB images are often "polluted" by ambient light, shadows, and complex textures that can mislead the algorithm.
The authors propose using the Infrared (IR) image already captured by the sensor. This offers two major advantages:
- Controlled Lighting: Most RGBD cameras use an internal IR projector. Treating this projected light as the primary source simplifies the shading analysis.
- Ambient Filtering: IR cameras typically use narrow-band filters to block visible light, making the system robust in indoor environments where overhead lights might otherwise interfere.
By fusing the initial, coarse depth map with high-frequency detail extracted from the IR shading information, the system produces a refined 3D point cloud.
Results and Qualitative Analysis
The method was evaluated using live data from an Intel RealSense F200 RGBD camera. The results demonstrate a "noticeable improvement" over raw sensor output.
As seen in the comparison, the raw output (top) appears "stair-stepped" and lacks definition on subtle surfaces. After applying the IR-based refinement (bottom), the surfaces appear significantly smoother, and fine-grained details that were previously lost in sensor noise become visible. Interestingly, the non-uniform nature of the projected IR pattern did not significantly degrade the shading-based refinement.
Discussion and Future Work
While promising, this preliminary work has limitations. The evaluation is currently qualitative; future research should include numerical metrics (such as Root Mean Square Error) against ground-truth laser scans. Additionally, the current approach assumes a simple lighting model. Explicitly modeling the highly non-uniform pattern of IR projectors could further enhance accuracy. Finally, the impact of material properties—such as shiny or dark objects that absorb or specularly reflect IR light—remains an area for further investigation.
Summary updated