Deep Cascade Generation on Point Sets

Kaiqi Wang, Ke Chen, Kui Jia

Abstract

Generating high-quality 3D point clouds from single images remains a challenge due to the permutation-insensitive nature of point sets. This paper introduces Deep Cascade Generation (DCG), a two-stage framework that first produces a coarse shape and then refines it using graph-based neighborhood encoding. By exploiting local geometric correlations, DCG captures intricate details and outperforms existing methods like AtlasNet with fewer parameters.

Introduction: The Challenge of 3D Point Sets

Generating 3D surfaces is a fundamental task in computer vision and graphics. While volumetric voxels were an early solution, their cubic memory cost limits scalability. Point clouds offer a more efficient alternative but lack explicit geometric connectivity. Standard deep learning models often struggle to exploit the underlying low-dimensional manifolds of an object, leading to noisy outputs that fail to capture fine topological details.

The Deep Cascade Generation (DCG) Framework

To address these limitations, the researchers proposed a two-stage "coarse-to-fine" approach. The DCG network breaks the generation process into a "sketch" and "detail" process, achieving better results with a significantly smaller parameter footprint.

Overview of the DCG architecture and qualitative comparison against AtlasNet.

Stage 1: Coarse Shape Generation

The first stage employs a modified version of AtlasNet to generate a sparse, coarse representation of the object. This stage focuses on establishing the correct global structure and basic proportions.

Stage 2: Local Shape Refinement

The core innovation lies in the refinement stage. The model constructs a $k$-nearest neighbor ($k$-NN) graph on the coarse points. A densely-connected graph encoder then mines local correlations between neighboring points, effectively "smoothing" the surface and ensuring geometric consistency relative to neighbors.

Ensemble of Decoders

Instead of a single large decoder, DCG utilizes an ensemble of smaller point decoders. This design helps the model avoid local minima and reduces the total number of parameters needed to generate a dense surface (approximately 60% fewer than AtlasNet).

Detailed view of the point decoder architecture using 2D primitives and 1D convolutions.

Dynamic Training Strategy

A "warm-up" strategy is implemented via a dynamic loss function. Early in training, the loss function prioritizes the coarse output; as training progresses, the weight shifts exponentially toward the refined, high-resolution output. This curriculum-based approach prevents the model from being distracted by fine details before mastering the basic proportions of the object.

Experimental Evaluation

The model was evaluated on the ShapeNet Core (v2) dataset, comprising nearly 40,000 CAD models across 13 categories. Tasks included single-view reconstruction and shape autoencoding.

Key Results

DCG demonstrated significant improvements over state-of-the-art methods like AtlasNet and PSG:

  • Quantitative Gains: DCG improved performance by 8.7% to 18.2% across various metrics on the ShapeNet benchmark.
  • Surface Fidelity: It achieved lower Chamfer Distance (CD) and Hausdorff Distance (HD), indicating higher proximity to ground-truth surfaces.
  • Qualitative Quality: Visual results show that DCG is superior at preserving thin structures, such as chair legs and airplane wings, while maintaining smoother surfaces compared to its predecessors.

Conclusion and Future Directions

DCG demonstrates that treating point clouds as graphs during refinement can drastically improve 3D reconstruction quality. While the model is highly efficient, future work could address the computational overhead of on-the-fly $k$-NN construction and explore dynamic point density for varying object complexities.

Summary updated

Abstract

This paper proposes a deep cascade network to generate 3D geometry of an object on a point cloud, consisting of a set of permutation-insensitive points. Such a surface representation is easy to learn from, but inhibits exploiting rich low-dimensional topological manifolds of the object shape due to lack of geometric connectivity. For benefiting from its simple structure yet utilizing rich neighborhood information across points, this paper proposes a two-stage cascade model on point sets. Specifically, our method adopts the state-of-the-art point set autoencoder to generate a sparsely coarse shape first, and then locally refines it by encoding neighborhood connectivity on a graph representation. An ensemble of sparse refined surface is designed to alleviate the suffering from local minima caused by modeling complex geometric manifolds. Moreover, our model develops a dynamically-weighted loss function for jointly penalizing the generation output of cascade levels at different training stages in a coarse-to-fine manner. Comparative evaluation on the publicly benchmarking ShapeNet dataset demonstrates superior performance of the proposed model to the state-of-the-art methods on both single-view shape reconstruction and shape autoencoding applications.
BibTeX
@inproceedings{wang2019deep,
  author = {Kaiqi Wang and Ke Chen and Kui Jia},
  title = {{Deep Cascade Generation on Point Sets}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/517},
}
Figure