Abstract
Generating realistic 3D objects is a well-established task for modern AI, yet providing fine-grained control over the specific topology of these objects remains a significant challenge. Most generative models operate as "black boxes," sampling from a learned distribution with little regard for specific structural requirements. We present StrucADT, a novel framework that introduces structural control into 3D point cloud generation. By utilizing a StructureGraph—a graph-based representation of part existence and adjacency—StrucADT allows users to define the blueprint of an object before generation. Our model combines Graph Attention Networks with a Diffusion Transformer to produce high-quality, diverse 3D shapes that strictly adhere to user-defined structural constraints.
Introduction
While current 3D generative models like PointFlow or DiffFacto can produce high-fidelity point clouds, they often lack the ability to specify how parts of an object should connect. For instance, a designer might require a chair where the armrests attach specifically to the backrest rather than the seat. Standard models struggle with such functional requirements because they do not explicitly learn the relationship between part existence and part adjacency. StrucADT addresses this by moving beyond global shape distributions toward structure-aware generation.
Methodology
The core innovation of StrucADT is the StructureGraph (SG), where nodes represent semantic parts (e.g., "legs," "backrest") and edges represent adjacency relationships. The architecture consists of three primary modules:
- StructureGraphNet (SGN): Utilizing a Graph Attention Network (GAT), this module processes the user's graph to extract structure-aware latent features ($Z$), ensuring the model understands the context of every connection.
- Structure-controlled Prior Module: To facilitate sampling, a conditional Continuous Normalizing Flow (cCNF) maps the complex distribution of shape features into a simple Gaussian distribution, conditioned on the part adjacency.
- Diffusion Transformer Module: The generation process uses a Diffusion Transformer (DiT) rather than a traditional U-Net. This allows the model to leverage cross-attention mechanisms to refer back to the StructureGraph and latent features during the denoising process, transforming a random point cloud into a coherent, structured shape.
Datasets and Evaluation
The model was evaluated on the ShapeNet and StructureNet datasets. For ShapeNet, the authors manually annotated adjacency relationships for thousands of shapes to create the necessary ground truth for the StructureGraph. Evaluation focused on both generative quality and structural fidelity.
Key Results
StrucADT demonstrates exceptional performance in maintaining structural consistency. The authors introduced a Structure Consistency Accuracy (SCA) metric, where StrucADT achieved scores between 80% and 98% across various categories, significantly outperforming state-of-the-art models like DiffFacto.
Qualitative analysis confirms that the model can successfully generate diverse configurations, such as chairs with armrests attached only to the back, only to the seat, or even floating if specified. Furthermore, the model exhibits strong structural generalization, producing plausible geometries for graph configurations not seen during training.
Conclusion and Future Work
StrucADT represents a significant step toward controllable 3D generation. By explicitly modeling part adjacency, it provides designers with a "blueprint" for AI-driven creation. Future work will focus on overcoming the data bottleneck of manual adjacency annotation through self-supervised learning and exploring multimodal control, such as text-to-structure-to-3D pipelines.
Summary updated