GenSDF: Two-Stage Learning of Generalizable Signed Distance Functions

Gene Chou, Ilya Chugunov, Felix Heide

Abstract

We investigate the generalization capabilities of neural signed distance functions (SDFs) for learning 3D object representations for unseen and unlabeled point clouds. Existing methods can fit SDFs to a handful of object classes and boast fine detail or fast inference speeds, but do not generalize well to unseen shapes. We introduce a two-stage semi-supervised meta-learning approach that transfers shape priors from labeled to unlabeled data to reconstruct unseen object categories. The first stage uses an episodic training scheme to simulate training on unlabeled data and meta-learns initial shape priors. The second stage then introduces unlabeled data with disjoint classes in a semi-supervised scheme to diversify these priors and achieve generalization. We assess our method on both synthetic data and real collected point clouds. Experimental results and analysis validate that our approach outperforms existing neural SDF methods and is capable of robust zero-shot inference on 100+ unseen classes.

Introduction

In 3D computer vision, representing complex geometries efficiently remains a significant challenge. Signed Distance Functions (SDFs) have emerged as a powerful representation, mapping spatial coordinates to the distance to the nearest surface—with the sign indicating whether a point is inside or outside the object. However, a persistent bottleneck exists: high-quality neural SDFs typically require dense, ground-truth supervision (signed distances), which is difficult to obtain for real-world scans. Conversely, unsupervised methods that work directly on raw point clouds often fail to generalize, producing "blobby" or inaccurate reconstructions when presented with object categories outside their narrow training set.

GenSDF, presented at NeurIPS 2022, addresses this gap by introducing a framework that learns a "language of shape." By leveraging a two-stage semi-supervised meta-learning approach, GenSDF bridges the divide between supervised precision and unsupervised scalability, enabling robust 3D reconstruction across hundreds of unseen object categories.

Methodology: A Two-Stage Approach

The core of GenSDF is a pipeline designed to cultivate a robust geometric prior that can adapt to novel shapes without explicit supervision.

Stage 1: Meta-Learning Shape Priors

The first stage employs an episodic training scheme. The model is trained on a labeled dataset partitioned into "support" (labeled) and "query" (pseudo-unlabeled) subsets. By forcing the model to reconstruct the query shapes using only self-supervised losses—while being guided by the ground-truth knowledge of the support set—the framework meta-learns how to extract geometric features from raw point clouds. This stage establishes a foundational "geometric gut feeling," allowing the model to understand basic structural properties before encountering diverse data.

Stage 2: Semi-Supervised Diversification

With the initial prior established, the second stage introduces a large volume of truly unlabeled data from disjoint object classes. This semi-supervised phase diversifies the model’s internal representation. Because the model has already learned how to learn geometry in Stage 1, it can successfully incorporate these new, unlabeled categories without the training process collapsing into trivial solutions.

The Signed Nearest Neighbor Loss

A critical technical innovation in GenSDF is the Signed Nearest Neighbor (SNN) loss. Traditional unsupervised methods often rely on unsigned distances, which struggle with thin structures or sharp edges because they lack orientation. The SNN loss utilizes the model’s own predicted sign to determine the direction to "pull" query points toward the surface. This disentanglement of absolute distance and sign allows GenSDF to recover fine details—such as the legs of a chair or the handle of a pitcher—that are typically lost in other generalizable models.

Experimental Evaluation

The authors evaluated GenSDF using both synthetic and real-world datasets:

  • Acronym Dataset: A large-scale subset of ShapeNet featuring over 8,800 models. The model was trained on 20 labeled and 76 unlabeled classes, then tested on 166 entirely unseen classes to evaluate zero-shot performance.
  • YCB Dataset: Real-world scans of common household objects used to test robustness against sensor noise and imperfect alignments.

Quantitative and Qualitative Results

GenSDF demonstrates a significant leap in generalization performance. On the 166 unseen classes of the Acronym dataset, GenSDF achieved a Chamfer Distance (CD) of 0.407. This vastly outperforms existing benchmarks like ConvOccNet (CD: 2.333) and the classic DeepSDF (CD: 33.19).

Qualitatively, the model excels at recovering thin structures and complex topologies. While previous methods often produce solid, undifferentiated masses for complex objects, GenSDF successfully reconstructs hollow interiors (like bowls) and delicate features (like the Stanford Bunny’s ears). Furthermore, the model proved robust to the noise inherent in real-world YCB scans, maintaining high fidelity where other methods failed.

Key Insights

  1. Diversity Over Quantity: Training on a diverse set of object classes (e.g., mixing furniture with tools) produces a significantly stronger prior for generalization than training on a larger volume of similar objects.
  2. The Necessity of Meta-Learning: Without the Stage 1 meta-learning phase, semi-supervised training is unstable. The meta-learning stage prevents the model from losing the "inside-outside" distinction when faced with noisy, unlabeled data.
  3. Zero-Shot Inference: The learned prior is sufficiently robust that the model can perform zero-shot inference on completely novel categories, producing plausible SDFs without any category-specific fine-tuning.

Conclusion and Future Work

GenSDF represents a major step toward general-purpose 3D shape representation. By combining meta-learning with semi-supervised diversification, it achieves unprecedented generalization across hundreds of object categories.

Future research directions include optimizing the model for real-time inference and extending the framework to handle partial-view point clouds (shape completion). Additionally, scaling GenSDF from single objects to complex, multi-object scenes remains a promising frontier for applications in robotics and autonomous navigation.

Summary updated

Abstract

We investigate the generalization capabilities of neural signed distance functions (SDFs) for learning 3D object representations for unseen and unlabeled point clouds. Existing methods can fit SDFs to a handful of object classes and boast fine detail or fast inference speeds, but do not generalize well to unseen shapes. We introduce a two-stage semi-supervised meta-learning approach that transfers shape priors from labeled to unlabeled data to reconstruct unseen object categories. The first stage uses an episodic training scheme to simulate training on unlabeled data and meta-learns initial shape priors. The second stage then introduces unlabeled data with disjoint classes in a semi-supervised scheme to diversify these priors and achieve generalization. We assess our method on both synthetic data and real collected point clouds. Experimental results and analysis validate that our approach outperforms existing neural SDF methods and is capable of robust zero-shot inference on 100+ unseen classes. Code can be found at https://github.com/princeton-computational-imaging/gensdf.
BibTeX
@inproceedings{chou2022gensdf,
  author = {Gene Chou and Ilya Chugunov and Felix Heide},
  title = {{GenSDF: Two-Stage Learning of Generalizable Signed Distance Functions}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=QK38rpF8RWL},
}
Figure