Introduction
Generating high-fidelity 3D volumetric shapes, typically represented as voxel grids, remains a fundamental challenge in computer vision. Traditional generative models often treat 3D shapes as monolithic entities, a "full-shape" approach that frequently results in the loss of fine-grained structural details. In these models, thin or complex components—such as the legs of a chair or the struts of a lamp—often appear blurred, disconnected, or structurally unsound.
To address these limitations, researchers have proposed part-based modeling, where shapes are assembled from individual semantic components. However, the "assembly" phase—determining the precise spatial relationship, scale, and orientation of these parts—is notoriously difficult. Previous attempts relied on simple regression or early feature merging, which often failed to capture the complex dependencies between parts. This paper introduces VoxAttention, a novel architecture that utilizes attention mechanisms to treat 3D modeling as a relational assembly task.
The VoxAttention Framework
The VoxAttention framework moves away from simple coordinate regression, instead employing a three-stage pipeline designed to preserve part identity and optimize spatial configuration.
1. Part Generation and Orthogonal Decomposition
The first stage involves an autoencoder that learns to decompose a complete shape into its constituent semantic parts. A critical innovation here is the use of mutual orthogonal projection matrices. By enforcing orthogonality in the latent space, the model ensures that the features of one part (e.g., a chair seat) do not "bleed" into another (e.g., the chair legs). This independence is vital for maintaining high-fidelity reconstruction during the later assembly phase.
2. Relational Assembly via Attention
The core contribution of the work is the replacement of standard Multi-Layer Perceptrons (MLPs) with an attention-based assembly module. The authors argue that assembly is fundamentally a relational problem; the position of a chair back is inherently dependent on the position and scale of the seat.
- Attention Mechanism: The network looks at the features of all parts simultaneously, allowing it to "negotiate" spatial boundaries and alignments.
- Channel-wise Attention: The model applies attention across feature channels, enabling it to focus on specific part characteristics across different layers of the decoder.
- Attention Consistency Loss: To ensure stability, the authors implemented a consistency loss term. This prevents the model from generating conflicting relational data as it processes information through multiple layers of the network.
3. End-to-End Fine-tuning
Finally, the entire system is refined end-to-end. This ensures that the generated parts and the learned assembly parameters work in concert to produce a cohesive and geometrically accurate final voxel grid.
Experimental Evaluation
The researchers evaluated VoxAttention using the ShapeNet Part dataset, focusing on four primary categories: Chairs, Airplanes, Lamps, and Guitars. All shapes were processed at a $32^3$ volumetric resolution.
Quantitative Results
VoxAttention demonstrated significant improvements over existing state-of-the-art (SOTA) methods. In the chair category, the model achieved a Shape mIoU (Mean Intersection over Union) of 78.4%. This represents a substantial leap over the simple MLP baseline (71.8%) and previous generative models like PQ-Net (67.3%).
Qualitative and Structural Integrity
Beyond raw accuracy, the model showed superior performance in maintaining structural symmetry. By utilizing relational attention, the model ensures that bilateral components—such as airplane wings or chair arms—are balanced in scale and correctly positioned relative to the central axis. Furthermore, the latent space proved robust enough to support "part swapping," where a user can replace one component (e.g., a specific chair back) and the model automatically adjusts the remaining parts to maintain a logical assembly.
Discussion and Future Work
While VoxAttention sets a new benchmark for volumetric assembly, certain challenges remain. The model occasionally struggles with "outlier" geometries—highly unconventional designs that are underrepresented in the training data. Additionally, while the authors demonstrated a $64^3$ resolution prototype, the computational cost of volumetric grids remains a hurdle for scaling to even higher resolutions.
The authors suggest that the principles of relational assembly and attention-based "gluing" of parts are not limited to voxels. Future research may involve applying these strategies to more efficient 3D representations, such as point clouds or meshes, to achieve even higher levels of geometric detail.
Summary updated