Topology-Aware Single-Image 3D Shape Reconstruction

Qimin Chen, Vincent Nguyen, Feng Han, Raimondas Kiveris, Zhuowen TuWorkshops @ CVPR 2020 · OpenReview

Abstract: We make an attempt to address topology-awareness for 3D shape reconstruction. Two types of high-level shape typologies are being studied here, namely genus (number of cuttings/holes) and connectivity (number of connected components), which are of great importance in 3D object reconstruction/understanding but have been thus far disjoint from the existing dense voxel-wise prediction literature. We propose a topology-aware shape autoencoder component (TPWCoder) by approximating topology property functions such as genus and connectivity with neural networks from the latent variables. TPWCoder can be directly combined with the existing 3D shape reconstruction pipelines for end-to-end training and prediction. On the challenging A Big CAD Model Dataset (ABC), TPWCoder demonstrates a noticeable quantitative and qualitative improvement over the competing methods, and it also shows improved quantitative result on the ShapeNet dataset.

Introduction

Traditional 3D shape reconstruction from a single 2D image typically relies on dense voxel-wise predictions. While effective for basic geometry, these "bottom-up" approaches often struggle with high-level structural properties, known as topology. For instance, a model might fail to reconstruct the hole in a handle or might produce "floating" noise voxels that aren't connected to the main object. These errors occur because standard loss functions, such as binary cross-entropy, treat each voxel independently and lack an understanding of global structure.

The TPWCoder Framework

To address these shortcomings, we propose TPWCoder (Topology-Aware Shape Autoencoder). The core innovation lies in incorporating topological constraints—specifically genus ($g$, the number of holes) and connectivity ($c$, the number of connected components)—into the learning process.

The TPWCoder architecture integrated with MarrNet, showing the latent vector z and the dual loss components: reconstruction loss and topology loss.

Because topological properties are discrete and non-differentiable, they cannot be directly optimized using gradient descent. TPWCoder overcomes this by using a neural network to approximate these functions. The model maps a 3D shape to a latent vector $z$, from which an auxiliary network predicts $g$ and $c$. This allows for a "topological loss" that guides the latent space to prioritize structural integrity during training.

Defining Topological Properties

The focus on genus and connectivity is particularly relevant for complex datasets like A Big CAD Model Dataset (ABC), where mechanical parts often feature intricate hole patterns and multiple components.

Examples of 3D objects from the ABC dataset with varying genus (g) and connectivity (c) values.

By forcing the latent representation to be predictive of these values, the network learns to suppress random noise (which would increase $c$) and maintain essential structural features like holes (which define $g$). This "top-down" guidance ensures that the bottleneck of the network preserves the most critical structural features of the object.

Results and Impact

The TPWCoder was evaluated against baseline models like MarrNet and ShapeHD. On the ABC dataset, the inclusion of topological priors led to significant quantitative improvements in Intersection over Union (IoU) and Chamfer Distance (CD). More importantly, it drastically reduced the Genus Number Error (GNE) and Connectivity Number Error (CNE) compared to models that did not account for topology.

Qualitatively, the reconstructed models are noticeably cleaner. The surfaces are smoother, and topological features like holes in mechanical brackets or chair backs are more accurately defined and placed. Even on the ShapeNet dataset, which features simpler topologies, the method remained highly competitive, demonstrating that topology awareness serves as a robust general prior for 3D reconstruction tasks.

Conclusion and Future Work

TPWCoder demonstrates that high-level geometric constraints can be effectively integrated into deep learning pipelines through differentiable approximation. While currently limited to voxel resolutions of $128^3$ and specific topological metrics, this work paves the way for more structurally sound 3D modeling. Future research may extend these principles to point clouds and meshes, where maintaining manifold properties is even more critical for downstream applications like engineering and manufacturing.

Summary updated

Abstract

We make an attempt to address topology-awareness for 3D shape reconstruction. Two types of high-level shape typologies are being studied here, namely genus (number of cuttings/holes) and connectivity (number of connected components), which are of great importance in 3D object reconstruction/understanding but have been thus far disjoint from the existing dense voxel-wise prediction literature. We propose a topology-aware shape autoencoder component (TPWCoder) by approximating topology property functions such as genus and connectivity with neural networks from the latent variables. TPWCoder can be directly combined with the existing 3D shape reconstruction pipelines for end-to-end training and prediction. On the challenging A Big CAD Model Dataset (ABC), TPWCoder demonstrates a noticeable quantitative and qualitative improvement over the competing methods, and it also shows improved quantitative result on the ShapeNet dataset.
BibTeX
@inproceedings{chen2020topology,
  author = {Qimin Chen and Vincent Nguyen and Feng Han and Raimondas Kiveris and Zhuowen Tu},
  title = {{Topology-Aware Single-Image 3D Shape Reconstruction}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=RIAb51Xz__z},
}
Figure