UCSG-NET- Unsupervised Discovering of Constructive Solid Geometry Tree

Kacper Kania, Maciej Zieba, Tomasz Kajdanowicz

Introduction

In the world of 3D modeling, there is a fundamental tension between accuracy and interpretability. State-of-the-art deep learning models can reconstruct complex 3D meshes with high precision using implicit representations like Signed Distance Fields (SDFs). However, these "black box" outputs are incredibly difficult for a human designer to edit in standard CAD software.

Constructive Solid Geometry (CSG) is the industry standard for 3D design because it represents shapes as a tree of simple primitives (e.g., cubes, spheres) combined via Boolean operations (Union, Intersection, Difference). Until now, teaching a computer to "reverse engineer" a CSG tree from a shape required supervised learning, meaning models needed a massive dataset of shapes paired with their exact underlying tree structures.

UCSG-NET addresses this limitation as the first neural network capable of discovering CSG trees in a completely unsupervised manner. By treating shape reconstruction as a differentiable tree-search problem, it transforms raw 3D data into interpretable, editable programs of primitives and Boolean operations without requiring ground-truth trees during training.

Methodology

The authors propose an end-to-end architecture that mimics the logic of a CAD engine while remaining differentiable so it can be trained via backpropagation.

UCSG-Net Architecture
  1. Primitive Prediction: An encoder-decoder network predicts the parameters (position, rotation, scale) for a set of simple geometric primitives.
  2. Differentiable Binarization: To perform Boolean operations, the model converts continuous SDF values into binary "occupancy" values (inside vs. outside). The authors employ a learnable clipping function ($\alpha$) that starts "fuzzy" to allow gradients to flow and gradually sharpens into a hard binary mask. This schedule avoids the vanishing gradient problem.
  3. CSG Layers & Gumbel-Softmax: The core innovation lies in the "CSG Layers," which must choose which two shapes to combine and which operation to use. Since "choosing" is a discrete, non-differentiable act, the model uses the Gumbel-Softmax trick. This allows the network to maintain a probability distribution over possible operations during training, eventually collapsing into a single, hard choice.
  4. Information Passing: A Gated Recurrent Unit (GRU) updates the latent representation as the tree is built, helping the model keep track of which parts of the shape still need to be reconstructed.

Datasets and Key Insights

The model was evaluated on two primary benchmarks: a 2D CAD Dataset (8,000 synthetic shapes consisting of chairs, desks, and lamps) and 3D ShapeNet (focusing on the 13 most common classes, voxelized at $64^3$ resolution).

Surprisingly, UCSG-NET outperformed the supervised version of CSG-NET on 2D tasks. This suggests that the "ground truth" trees in existing datasets might actually be suboptimal, and an unsupervised model can find more efficient ways to represent geometry. Qualitatively, the model naturally learns to reconstruct semantic parts separately—for example, building an airplane's wings and hull as distinct branches of the tree before uniting them in the final layer.

Key Results

  • 2D Reconstruction: On the CAD dataset, UCSG-NET achieved a Chamfer Distance (CD) of 0.32, significantly outperforming the supervised CSG-NET (CD 0.39) and its Reinforcement Learning variants.
  • 3D Reconstruction: In 3D tasks, UCSG-NET (CD 2.085) outperformed other interpretable baselines like Visual Primitives (CD 2.259).
2D CSG Tree Reconstructions

While UCSG-NET did not beat non-interpretable methods like BSP-Net (CD 0.446) in pure reconstruction accuracy, the authors argue that the trade-off is highly valuable. UCSG-NET produces a literal tree of operations that can be imported directly into CAD tools for human editing, whereas BSP-Net produces a static, uneditable mesh.

Limitations and Future Work

Despite its advances, UCSG-NET has limitations. The model struggles with very fine details (e.g., engines on airplane wings) due to the limited number of primitives and layers it can handle before the tree becomes computationally too expensive. Furthermore, because there are often infinite ways to construct the same shape using CSG, the model doesn't always choose the most "human-logical" path, sometimes using overlapping unions to pass a primitive through a layer untouched.

Future directions suggested by the authors include incorporating "weak supervision" (providing hints about which operations to use) or using RANSAC to provide better initial guesses for the primitive shapes, further bridging the gap between raw geometry and editable CAD models.

Summary updated

Abstract

Signed distance field (SDF) is a prominent implicit representation of 3D meshes. Methods that are based on such representation achieved state-of-the-art 3D shape reconstruction quality. However, these methods struggle to reconstruct non-convex shapes. One remedy is to incorporate a constructive solid geometry framework (CSG) that represents a shape as a decomposition into primitives. It allows to embody a 3D shape of high complexity and non-convexity with a simple tree representation of Boolean operations. Nevertheless, existing approaches are supervised and require the entire CSG parse tree that is given upfront during the training process. On the contrary, we propose a model that extracts a CSG parse tree without any supervision - UCSG-Net. Our model predicts parameters of primitives and binarizes their SDF representation through differentiable indicator function. It is achieved jointly with discovering the structure of a Boolean operators tree. The model selects dynamically which operator combination over primitives leads to the reconstruction of high fidelity. We evaluate our method on 2D and 3D autoencoding tasks. We show that the predicted parse tree representation is interpretable and can be used in CAD software.
BibTeX
@inproceedings{kania2020ucsg,
  author = {Kacper Kania and Maciej Zieba and Tomasz Kajdanowicz},
  title = {{UCSG-NET- Unsupervised Discovering of Constructive Solid Geometry Tree}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=DbLRVNLf1Jp},
}
Figure