ShapeMorph: 3D Shape Completion via Blockwise Discrete Diffusion

Jiahui Li, Pourya Shamsolmoali, Yue Lu, Masoumeh ZareapoorWACV 2025 · DOI

Introduction

3D scanning in real-world environments often yields incomplete point clouds due to sensor limitations, occlusions, and restricted viewing angles. Completing these shapes is a fundamental challenge in 3D vision, complicated by three primary issues: the inherent ambiguity of partial data, the error accumulation characteristic of autoregressive (AR) models, and the difficulty in maintaining fine-grained topological fidelity. Existing generative approaches frequently produce "blobby" or disconnected geometries that fail to capture the structural nuances of the original objects.

Methodology

ShapeMorph addresses these challenges through a novel framework that combines irregular discrete representations with a blockwise diffusion process. The pipeline consists of three main stages: shape quantification, blockwise discrete diffusion, and latent completion.

Irregular Discrete Representation

Unlike traditional grid-based methods that rely on rigid voxels, ShapeMorph encodes 3D shapes into sequences of 4-tuples $(v, x, y, z)$. In this representation, $v$ is a discrete codebook index representing a local geometric pattern (learned via VQVAE), while $x, y, z$ are the quantified spatial coordinates of that pattern. This irregular distribution allows the model to capture surface details more efficiently and accurately than a fixed grid.

Blockwise Discrete Diffusion Model (BDDM)

To prevent the model from confusing different attribute types—such as swapping a spatial coordinate for a geometric pattern index—ShapeMorph employs a block-diagonal transition matrix. This ensures that during the denoising process, coordinate tokens are only compared against other coordinates, and codebook indices against other indices, respecting the semantic boundaries of the multi-attribute sequence.

ShapeMorph Pipeline Overview

Flow Transformer (FLOT)

To handle the long sequences generated by complex 3D shapes, the authors designed the Flow Transformer (FLOT). It utilizes a specialized "switch operation" that alternates between attribute-level attention (modeling dependencies within a category, like all x-coordinates) and tuple-level attention (modeling relationships within a single local patch). This dual-path approach captures both global structure and local intricacies without the computational explosion of standard self-attention.

Experiments and Results

The model was evaluated on major benchmarks including ShapeNet, PartNet, GenRe, and RedWood. ShapeMorph consistently outperformed state-of-the-art (SOTA) baselines such as SFormer, AutoSDF, and PVD in both fidelity and diversity.

Quantitative Performance

On the ShapeNet Random Scan task, ShapeMorph achieved a Minimal Matching Distance (MMD) of 1.33, significantly outperforming SFormer's 1.89. Additionally, the model is approximately 3.6x faster at inference than its primary competitors and reduces parameter complexity by 84% compared to grid-based models like 3DQD.

Qualitative Analysis

Visual results demonstrate ShapeMorph's ability to generate clean surfaces and realistic part junctions (e.g., where a chair leg meets the seat). While previous methods often produce disconnected or "adhesive" parts, ShapeMorph maintains sharp topological boundaries and generates a wider variety of plausible completions for ambiguous inputs.

Qualitative comparison on ShapeNet

Conclusion and Future Work

ShapeMorph introduces a robust framework for 3D shape completion by leveraging irregular discrete representations and blockwise diffusion. By respecting semantic boundaries between attributes and refining the entire shape simultaneously, it overcomes the drift issues of AR models. While diffusion remains inherently slower than single-pass models, the efficiency gains of FLOT make it a highly competitive solution. Future work may extend this blockwise approach to multi-modal tasks such as text-to-shape generation.

Summary updated

BibTeX
@inproceedings{li2025shapemorph,
  author = {Jiahui Li and Pourya Shamsolmoali and Yue Lu and Masoumeh Zareapoor},
  title = {{ShapeMorph: 3D Shape Completion via Blockwise Discrete Diffusion}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00279},
}
Figure