SfmCAD: Unsupervised CAD Reconstruction by Learning Sketch-based Feature Modeling Operations

Pu Li, Jianwei Guo, Huibin Li, Bedrich Benes, Dong-Ming YanCVPR 2024 · DOI

Introduction

In industrial design and manufacturing, 3D shapes are rarely stored as raw point clouds or meshes. Instead, engineers utilize Computer-Aided Design (CAD) models, which consist of sequences of geometric operations—such as "draw this circle and extrude it five inches." These models are highly valued because they are compact, interpretable, and, most importantly, easy to edit.

However, converting a raw 3D scan or voxel grid back into a structured CAD program is a significant challenge. Most existing deep learning approaches either require massive amounts of labeled "design history" data—which is scarce—or rely on simple geometric primitives like spheres and boxes that fail to capture intricate details. SfmCAD (Sketch-from-CAD) addresses this by introducing an unsupervised framework that "reverse engineers" the design process, turning messy 3D shapes into clean, parametric CAD representations.

Methodology: The Sketch-and-Path Paradigm

The core innovation of SfmCAD is the "typed sketch+path" representation. Rather than attempting to predict a dense 3D volume directly, the network learns to decompose an object into a 2D cross-section (the sketch) and a 3D trajectory (the path).

Standard CAD operations like Extrude, Loft, Sweep, and Revolve can be unified under a path-based modeling framework.

To ensure stability and avoid "tangled" geometry, the system employs a two-stage, coarse-to-fine learning strategy:

  1. Stage 1: Box+Path Learning: The network first identifies the primary "skeleton" of the object. It predicts a 3D Bézier curve (the path) and places a series of rough bounding boxes along it. This captures the general sweep and orientation of the shape.
  2. Stage 2: Implicit Sketch Learning: Once the path is established, the network focuses on geometric details. It uses an implicit neural network to define the 2D Signed Distance Function (SDF) of the cross-section. This allows the model to represent complex, non-primitive shapes for the sketches.

Bézier curves serve as a unified geometric language within this framework: a straight line represents an Extrude, a curved trajectory a Sweep, and a circular path a Revolve. To make this work without supervision, the authors developed differentiable CAD operators. This allows the network to "render" its predicted CAD program back into a 3D shape, compare it to the original input, and use the reconstruction error to update its parameters via backpropagation.

Experimental Results

The researchers evaluated SfmCAD across several diverse datasets, including the ABC Dataset (one million engineering-grade CAD parts), ShapeNet (standard furniture benchmarks), and a custom Tree Dataset designed to test the model's ability to handle complex, thin, and curving structures.

SfmCAD reconstructs diverse 3D shapes into editable CAD-like structures by decomposing them into paths and sketches.

SfmCAD consistently outperformed existing unsupervised methods in both accuracy and visual quality. On the ABC dataset, SfmCAD achieved a Chamfer Distance (CD) of 0.395, significantly lower than the next best competitor, SECAD-Net (0.506). It also demonstrated superior Normal Consistency, meaning the surfaces it reconstructs are smoother and more physically plausible. Visual results confirm that SfmCAD is far better at capturing thin structures—like the spindly legs of a chair or the branches of a tree—that primitive-based methods usually ignore or distort.

Discussion and Future Work

By decoupling global structure (the path) from local detail (the sketch), SfmCAD provides a robust geometric prior that guides the reconstruction process. Because the output consists of Bézier control points and 2D sketches, the results are inherently interpretable; a human user can manually tweak the resulting CAD program, a feat impossible with standard "black box" 3D reconstructions.

While a major step forward, the model currently assumes relatively continuous sweeps. Future work will explore integrating subtractive boolean operations (like drilling holes) and utilizing 2D sketch templates to further enhance the robustness and industrial utility of the framework.

Summary updated

BibTeX
@inproceedings{li2024sfmcad,
  author = {Pu Li and Jianwei Guo and Huibin Li and Bedrich Benes and Dong-Ming Yan},
  title = {{SfmCAD: Unsupervised CAD Reconstruction by Learning Sketch-based Feature Modeling Operations}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.00447},
}
Figure