Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction

Sijia Jiang, Jing Hua, Zhizhong HanICCV 2023 · OpenReview

Abstract

In recent years, huge progress has been made on learning neural implicit representations from multi-view images for 3D reconstruction. As an additional input complementing coordinates, using sinusoidal functions as positional encodings plays a key role in revealing high frequency details with coordinate-based neural networks. However, high frequency positional encodings make the optimization unstable, which results in noisy reconstructions and artifacts in empty space. To resolve this issue in a general sense, we introduce to learn neural implicit representations with quantized coordinates, which reduces the uncertainty and ambiguity in the field during optimization. Instead of continuous coordinates, we discretize continuous coordinates into discrete coordinates using nearest interpolation among quantized coordinates which are obtained by discretizing the field in an extremely high resolution. We use discrete coordinates and their positional encodings to learn implicit functions through volume rendering. This significantly reduces the variations in the sample space, and triggers more multi-view consistency constraints on intersections of rays from different views, which enables to infer implicit function in a more effective way. Our quantized coordinates do not bring any computational burden, and can seamlessly work upon the latest methods. Our evaluations under the widely used benchmarks show our superiority over the state-of-the-art.

Introduction

Neural implicit representations, such as Neural Radiance Fields (NeRF) and Surface Reconstruction (NeuS), have transformed 3D reconstruction by mapping spatial coordinates to scene properties via neural networks. To capture high-frequency geometric details, these methods typically employ positional encodings (PE). While effective, PE introduces significant optimization challenges. High-frequency signals make the network hypersensitive to input variations, leading to "floaters" in empty space and noisy surface reconstructions. Furthermore, in a continuous coordinate space, rays from different viewpoints are statistically unlikely to sample the exact same 3D point, hindering the enforcement of multi-view consistency.

Methodology: Coordinate Quantization

The proposed Coordinate Quantized Neural Implicit Representation (CQ-NIR) addresses these issues by discretizing the continuous 3D space into an extremely high-resolution grid (e.g., $51200^3$). During training, every sampled coordinate along a camera ray is snapped to its nearest grid center—a "quantized coordinate"—before being processed by the positional encoding layer.

Comparison of continuous (a) and discrete (b) coordinate pipelines for neural implicit representations.

This quantization acts as a "plug-and-play" preprocessing step that can be integrated into existing frameworks like UNISURF or NeuS without architectural changes. To maintain the integrity of volume rendering, duplicate quantized points along a single ray are removed, ensuring that the integral correctly represents the physical space.

Enhancing Multi-view Consistency

A core advantage of CQ-NIR is its ability to force multi-view consistency. In continuous space, rays from different cameras often "miss" each other by infinitesimal distances, preventing the model from recognizing that they are observing the same physical location. By voxelizing the space, these rays are much more likely to intersect within the same discrete "box," triggering explicit consistency constraints.

Conceptual illustration of how discrete coordinates (b) trigger multi-view consistency constraints at ray intersections compared to continuous coordinates (a).

Experimental Results

Evaluations on the DTU, ScanNet, and Replica datasets demonstrate that CQ-NIR consistently improves reconstruction accuracy and stability. On the DTU benchmark, the method reduced the mean Chamfer distance for UNISURF from 1.02 to 0.80. Quantitatively, the discrete model observed 86.2 times fewer unique points while utilizing 34 times more multi-view constraints at intersections. This reduction in the search space leads to significantly faster convergence and the elimination of high-frequency "salt-and-pepper" noise.

Conclusion

CQ-NIR provides a simple yet effective strategy for stabilizing neural implicit representation training. By discretizing the coordinate space, it effectively bridges the gap between continuous neural fields and discrete geometric consistency, leading to cleaner, more accurate 3D reconstructions.

Summary updated

Abstract

In recent years, huge progress has been made on learning neural implicit representations from multi-view images for 3D reconstruction. As an additional input complementing coordinates, using sinusoidal functions as positional encodings plays a key role in revealing high frequency details with coordinate-based neural networks. However, high frequency positional encodings make the optimization unstable, which results in noisy reconstructions and artifacts in empty space. To resolve this issue in a general sense, we introduce to learn neural implicit representations with quantized coordinates, which reduces the uncertainty and ambiguity in the field during optimization. Instead of continuous coordinates, we discretize continuous coordinates into discrete coordinates using nearest interpolation among quantized coordinates which are obtained by discretizing the field in an extremely high resolution. We use discrete coordinates and their positional encodings to learn implicit functions through volume rendering. This significantly reduces the variations in the sample space, and triggers more multi-view consistency constraints on intersections of rays from different views, which enables to infer implicit function in a more effective way. Our quantized coordinates do not bring any computational burden, and can seamlessly work upon the latest methods. Our evaluations under the widely used benchmarks show our superiority over the state-of-the-art. Our code is available at https://github.com/MachinePerceptionLab/CQ-NIR.
BibTeX
@inproceedings{jiang2023coordinate,
  author = {Sijia Jiang and Jing Hua and Zhizhong Han},
  title = {{Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  url = {https://openreview.net/forum?id=eNlGnlF807},
}
Figure