StrucADT: Generating Structure-controlled 3D Point Clouds with Adjacency Diffusion Transformer

Zhenyu Shu, Jiajun Shen, Zhongui Chen, Xiaoguang Han, Shiqing Xin

Abstract

Generating realistic 3D objects is a well-established task for modern AI, yet providing fine-grained control over the specific topology of these objects remains a significant challenge. Most generative models operate as "black boxes," sampling from a learned distribution with little regard for specific structural requirements. We present StrucADT, a novel framework that introduces structural control into 3D point cloud generation. By utilizing a StructureGraph—a graph-based representation of part existence and adjacency—StrucADT allows users to define the blueprint of an object before generation. Our model combines Graph Attention Networks with a Diffusion Transformer to produce high-quality, diverse 3D shapes that strictly adhere to user-defined structural constraints.

Introduction

While current 3D generative models like PointFlow or DiffFacto can produce high-fidelity point clouds, they often lack the ability to specify how parts of an object should connect. For instance, a designer might require a chair where the armrests attach specifically to the backrest rather than the seat. Standard models struggle with such functional requirements because they do not explicitly learn the relationship between part existence and part adjacency. StrucADT addresses this by moving beyond global shape distributions toward structure-aware generation.

Methodology

The core innovation of StrucADT is the StructureGraph (SG), where nodes represent semantic parts (e.g., "legs," "backrest") and edges represent adjacency relationships. The architecture consists of three primary modules:

  1. StructureGraphNet (SGN): Utilizing a Graph Attention Network (GAT), this module processes the user's graph to extract structure-aware latent features ($Z$), ensuring the model understands the context of every connection.
  2. Structure-controlled Prior Module: To facilitate sampling, a conditional Continuous Normalizing Flow (cCNF) maps the complex distribution of shape features into a simple Gaussian distribution, conditioned on the part adjacency.
  3. Diffusion Transformer Module: The generation process uses a Diffusion Transformer (DiT) rather than a traditional U-Net. This allows the model to leverage cross-attention mechanisms to refer back to the StructureGraph and latent features during the denoising process, transforming a random point cloud into a coherent, structured shape.
The StrucADT architecture, featuring the Diffusion Transformer, StructureGraphNet, and Prior modules.

Datasets and Evaluation

The model was evaluated on the ShapeNet and StructureNet datasets. For ShapeNet, the authors manually annotated adjacency relationships for thousands of shapes to create the necessary ground truth for the StructureGraph. Evaluation focused on both generative quality and structural fidelity.

Key Results

StrucADT demonstrates exceptional performance in maintaining structural consistency. The authors introduced a Structure Consistency Accuracy (SCA) metric, where StrucADT achieved scores between 80% and 98% across various categories, significantly outperforming state-of-the-art models like DiffFacto.

Qualitative results showing how different StructureGraph configurations lead to distinct, topologically accurate 3D chair generations.

Qualitative analysis confirms that the model can successfully generate diverse configurations, such as chairs with armrests attached only to the back, only to the seat, or even floating if specified. Furthermore, the model exhibits strong structural generalization, producing plausible geometries for graph configurations not seen during training.

Conclusion and Future Work

StrucADT represents a significant step toward controllable 3D generation. By explicitly modeling part adjacency, it provides designers with a "blueprint" for AI-driven creation. Future work will focus on overcoming the data bottleneck of manual adjacency annotation through self-supervised learning and exploring multimodal control, such as text-to-structure-to-3D pipelines.

Summary updated

Abstract

In the field of 3D point cloud generation, numerous 3D generative models have demonstrated the ability to generate diverse and realistic 3D shapes. However, the majority of these approaches struggle to generate controllable 3D point cloud shapes that meet user-specific requirements, hindering the large-scale application of 3D point cloud generation. To address the challenge of lacking control in 3D point cloud generation, we are the first to propose controlling the generation of point clouds by shape structures that comprise part existences and part adjacency relationships. We manually annotate the adjacency relationships between the segmented parts of point cloud shapes, thereby constructing a StructureGraph representation. Based on this StructureGraph representation, we introduce StrucADT, a novel structure-controllable point cloud generation model, which consists of StructureGraphNet module to extract structure-aware latent features, cCNF Prior module to learn the distribution of the latent features controlled by the part adjacency, and Diffusion Transformer module conditioned on the latent features and part adjacency to generate structure-consistent point cloud shapes. Experimental results demonstrate that our structure-controllable 3D point cloud generation method produces high-quality and diverse point cloud shapes, enabling the generation of controllable point clouds based on user-specified shape structures and achieving state-of-the-art performance in controllable point cloud generation on the ShapeNet dataset.
BibTeX
@article{shu2025strucadt,
  author = {Zhenyu Shu and Jiajun Shen and Zhongui Chen and Xiaoguang Han and Shiqing Xin},
  title = {{StrucADT: Generating Structure-controlled 3D Point Clouds with Adjacency Diffusion Transformer}},
  journal = {IEEE Trans. Vis. Comput. Graph.},
  year = {2025},
}
Figure