PAPRec: 3D Point Cloud Reconstruction Based on Prior-Guided Adaptive Probabilistic Network

Caixia Liu, Minhong Zhu, Yali Chen, Xiulan Wei, Haisheng LiSensors 2025 · DOI

TL;DR

This paper introduces PAPRec, a novel framework for reconstructing complete 3D point clouds from a single 2D image. By integrating 3D prior knowledge and an adaptive probabilistic decoder, PAPRec significantly improves reconstruction accuracy and detail compared to existing methods, addressing the inherent ambiguity of single-view 3D reconstruction.

What Problem Does This Paper Address?

Reconstructing a complete 3D object from a single 2D image is a notoriously challenging task in computer vision. This is because a single image inherently lacks depth information and can be ambiguous, leading to incomplete or inaccurate 3D models. Existing methods often struggle with insufficient feature representation, unstable training, and limited constraints, resulting in low accuracy and models that miss crucial details or contain noise. The goal is to bridge this gap by developing a robust method that can infer a full 3D shape from just one viewpoint.

Methods

PAPRec tackles this problem with a multi-stage approach:

  • Prior Feature Extraction: The system first encodes both the input single-view image and its corresponding 3D point cloud (during training) into feature distributions. This step leverages the idea that a 3D prior can guide the reconstruction process.
  • Latent Shape Learning: A latent normalizing flow model is employed to learn a mapping between the image and point cloud feature distributions. This process distills rich information into a compact shape latent vector, capturing global object characteristics.
  • Adaptive Probabilistic Decoding: This is the core innovation. PAPRec uses an adaptive probabilistic decoder that combines two powerful generative models:
  • Shape Normalizing Flow: This model is responsible for capturing the overall distribution of the point cloud, providing global shape information.
  • Diffusion Model: This model focuses on extracting local details and refining the reconstruction, addressing finer geometric features. Crucially, these two models are combined with adaptive weights, allowing the network to dynamically balance global and local feature learning based on the input. The entire network is trained end-to-end using a combined loss function that includes prior, diffusion, and flow losses.

Datasets

The primary dataset used for training and evaluation is ShapeNet. For the 3D generation task, a large subset of 51,127 shapes from 55 categories was utilized. For the single-view 3D reconstruction task, 13 specific object categories from ShapeNet were used, with data pairs of images and corresponding point clouds. The datasets were split into 80% for training and 20% for testing.

Key Insights and Ideas

  • 3D Priors are Crucial: The paper strongly advocates for the use of 3D prior information. By learning from both the image and a corresponding 3D shape during training, the model gains a richer understanding of object geometry, which is essential for overcoming the ambiguity of single-view input.
  • Adaptive Decoding is Key: The adaptive combination of a normalizing flow (for global structure) and a diffusion model (for local details) in the decoder is a significant advancement. This allows the model to flexibly leverage the strengths of each generative approach, leading to more comprehensive and accurate reconstructions.
  • Bridging Global and Local: The method effectively addresses the challenge of capturing both the overall shape and fine-grained details. The normalizing flow handles the broad structure, while the diffusion model refines the output, preventing the loss of critical geometric information.

Key Results

PAPRec demonstrates strong performance on the ShapeNet dataset:

  • Quantitative Improvements: On average, PAPRec achieved significant improvements over state-of-the-art methods, including:
  • Chamfer Distance (CD): 2.62% improvement.
  • Earth Mover's Distance (EMD): 5.99% improvement.
  • F1-score: 4.41% improvement.
  • Qualitative Results: Visual comparisons show that PAPRec reconstructs more complete and detailed 3D point clouds. For instance, reconstructed chairs and airplanes exhibit higher accuracy, and lamps are more complete. The method also shows better robustness in handling challenging cases like self-occlusion.

Limitations and Future Work

  • Generating Fine Details: While PAPRec shows substantial improvements, the authors acknowledge that there is still room for enhancement in generating extremely fine details.
  • Future Directions: The authors plan to conduct further research on improving reconstruction accuracy by exploring prior-based diffusion models. This suggests a continued focus on leveraging generative models and prior information for more precise 3D reconstruction, particularly for applications requiring high-fidelity models.

Summary updated

Abstract

Inferring a complete 3D shape from a single-view image is an ill-posed problem. The proposed methods often have problems such as insufficient feature expression, unstable training and limited constraints, resulting in a low accuracy and ambiguity reconstruction. To address these problems, we propose a prior-guided adaptive probabilistic network for single-view 3D reconstruction, called PAPRec. In the training stage, PAPRec encodes a single-view image and its corresponding 3D prior into image feature distribution and point cloud feature distribution, respectively. PAPRec then utilizes a latent normalizing flow to fit the two distributions and obtains a latent vector with rich cues. PAPRec finally introduces an adaptive probabilistic network consisting of a shape normalizing flow and a diffusion model in order to decode the latent vector as a complete 3D point cloud. Unlike the proposed methods, PAPRec fully learns the global and local features of objects by innovatively integrating 3D prior guidance and the adaptive probability network under the optimization of a loss function combining prior, flow and diffusion losses. The experimental results on the public ShapeNet dataset show that PAPRec, on average, improves CD by 2.62%, EMD by 5.99% and F1 by 4.41%, in comparison to several state-of-the-art methods.
BibTeX
@inproceedings{liu2025paprec,
  author = {Caixia Liu and Minhong Zhu and Yali Chen and Xiulan Wei and Haisheng Li},
  title = {{PAPRec: 3D Point Cloud Reconstruction Based on Prior-Guided Adaptive Probabilistic Network}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25051354},
}
Figure