DQRNet: Dynamic Quality Refinement Network for 3D Reconstruction from a Single Depth View

Caixia Liu, Minhong Zhu, Haisheng Li, Xiulan Wei, Lianqing Li, Qianwen YaoSensors 2025 · DOI

TL;DR

This paper introduces DQRNet, a novel network for reconstructing complete and detailed 3D shapes from a single depth image. By employing a dynamic encoder-decoder with a unique latent extractor and a two-stage refinement process (global and local), DQRNet significantly improves accuracy and robustness, particularly at object boundaries, outperforming existing state-of-the-art methods.

What Problem Does This Paper Address?

3D reconstruction from depth images is crucial for many applications like SLAM, virtual reality, and autonomous driving. However, real-world depth scans often suffer from self-occlusion and environmental occlusion, leading to incomplete shapes with lost details. Existing methods struggle to recover these missing parts accurately and often produce smooth, undetailed reconstructions. This paper aims to bridge this gap by developing a method that can generate high-resolution, detailed 3D shapes from a single, potentially incomplete, depth view.

Methods

DQRNet is built upon two main components:

  • Dynamic Encoder-Decoder: This part takes a single depth view and transforms it into a 2.5D voxel grid. The encoder then processes this grid to extract features. The novelty here lies in the dynamic latent extractor, which adaptively selects the most relevant latent information from the encoder's output. This is achieved by using two neural networks: one to predict weights for each latent feature and another to determine a threshold. This process helps to reduce noise and improve robustness by focusing on important features. The decoder then reconstructs a coarse 3D shape.
  • Detail Quality Refiner: This component takes the coarse 3D shape and further refines it to capture fine details. It consists of two parts:
  • Global Point Refiner: This uses a discriminator with 3D convolutional layers and multi-head self-attention to capture long-range dependencies and improve the overall global structure of the shape.
  • Local Point Refiner: This module specifically targets uncertain regions (often at object boundaries) by sampling points with high uncertainty. It then extracts multi-scale features and uses a weighted fusion strategy to combine pre- and post-predicted points, thereby refining boundary details.

Datasets

The authors used the ShapeNet dataset for training and evaluation. The dataset consists of 3D models and their corresponding depth views. For experiments, they utilized a training set with 213 CAD models across four categories (bench, chair, couch, table) and a testing set with 37 CAD models. They generated training and testing pairs by capturing models from various viewpoints.

Key Insights and Ideas

  • Adaptive Feature Selection is Key: The core idea of the "dynamic latent extractor" is that not all features extracted from a depth map are equally important for 3D reconstruction. By adaptively selecting the most informative latent features, the model can better handle noise and improve robustness.
  • Two-Stage Refinement for Detail: The separation of global structure refinement (via the global refiner) and fine-grained boundary detail enhancement (via the local refiner) is crucial. This allows the network to address both the overall shape and the intricate details that are often lost in single-view reconstruction.
  • Focusing on Uncertainty: The local point refiner's strategy of specifically targeting "uncertain points" is a smart way to address the inherent challenges of occlusion and incomplete data in depth maps. By focusing refinement efforts where they are most needed, the model can achieve better results.

Key Results

  • Improved Accuracy: DQRNet achieved state-of-the-art results on the ShapeNet dataset. It improved the average Intersection over Union (IoU) by 2.09% and reduced the average Chamfer Error (CE) by 2.88% compared to other methods.
  • Quantitative Performance: In per-category experiments, DQRNet consistently outperformed other methods in both IoU and CE loss for both same-view and cross-view testing. For instance, on cross-view testing, it showed an average IoU improvement of 2.23% over 3D-RVP.
  • Qualitative Improvements: Visual results demonstrated that DQRNet produces more complete shapes with finer details, especially at object boundaries, compared to methods like SeedFormer (which produced fragmented shapes) and SnowflakeNet (which had less delicate structures).
  • Efficiency Balance: While DQRNet has slightly more parameters and flops than 3D-RVP, its inference speed is comparable, suggesting an effective balance between performance and computational cost due to its selective point processing.

Limitations and Future Work

  • Limitations: The paper notes that same-view reconstruction results are better than cross-view results. This is attributed to the model having limited information when encountering unseen views, making it harder to predict missing regions.
  • Future Work: The authors plan to continue optimizing DQRNet's performance and explore its application in more challenging 3D reconstruction scenarios. They also intend to investigate advanced techniques like diffusion models and NeRF to achieve even more efficient and accurate 3D reconstruction.

Summary updated

Abstract

With the widespread adoption of 3D scanning technology, depth view-driven 3D reconstruction has become crucial for applications such as SLAM, virtual reality, and autonomous vehicles. However, due to the effects of self-occlusion and environmental occlusion, obtaining complete and error-free 3D shapes directly from 3D scans remains challenging, as previous reconstruction methods tend to lose details. To this end, we propose Dynamic Quality Refinement Network (DQRNet) for reconstructing complete and accurate 3D shape from a single depth view. DQRNet introduces a dynamic encoder-decoder and a detail quality refiner to generate high-resolution 3D shapes, where the former designs a dynamic latent extractor to adaptively select important parts of an object and the latter designs global and local point refiners to enhance the reconstruction quality. Experimental results show that DQRNet is able to focus on capturing the details at boundaries and key areas on ShapeNet dataset, thereby achieving better accuracy and robustness than SOTA methods.
BibTeX
@inproceedings{liu2025dqrnet,
  author = {Caixia Liu and Minhong Zhu and Haisheng Li and Xiulan Wei and Lianqing Li and Qianwen Yao},
  title = {{DQRNet: Dynamic Quality Refinement Network for 3D Reconstruction from a Single Depth View}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25051503},
}
Figure