Introduction
Generating and manipulating 3D meshes based on high-level semantic attributes—such as "weight" for human bodies or "happiness" for facial expressions—is a significant challenge in computer graphics. Traditional tools often require manual vertex manipulation, which is labor-intensive for digital artists. Rank3DGAN addresses this by combining a multi-chart mesh representation with a ranking-based generative adversarial network (GAN), allowing for intuitive control over complex 3D shapes using relative concepts rather than absolute numerical labels.
Problem Statement
Existing deep learning approaches for 3D generation face several hurdles:
- Voxel-based models are memory-intensive and often yield low-resolution, "blocky" results.
- Point clouds lack the surface connectivity necessary for smooth mesh reconstruction.
- Data Annotation is notoriously difficult for subjective traits. While humans struggle to assign absolute scores (e.g., "How happy is this face on a scale of 1 to 100?"), they excel at relative comparisons (e.g., "Face A is happier than Face B").
Methodology
Rank3DGAN introduces two core innovations: a multi-chart representation and a ranking-based GAN architecture.
Multi-chart Mesh Representation
To leverage standard 2D Convolutional Neural Networks (CNNs), the framework "unwraps" 3D meshes into a set of 2D images, or "charts," using orbifold Tutte embedding. This mapping is bijective, meaning the 2D images can be perfectly reconstructed into a 3D surface. This allows the GAN to generate "images" that implicitly represent 3D geometry while maintaining surface connectivity.
Rank3DGAN Architecture
The architecture consists of a generator and a critic with dual roles. The generator takes a noise vector ($z$) and a target attribute value ($r$) to produce the 2D charts. The critic determines if a mesh is real or fake and acts as an auxiliary ranker. By training on pairs of meshes with known relative orderings, the ranking loss forces the network to organize its latent space such that higher values of $r$ correspond to "more" of a specific attribute.
Results and Discussion
The model was evaluated on diverse datasets including MPII Human Shape, CUB-200-2011 birds, and the Basel Face Model. Rank3DGAN significantly outperformed baselines like Voxel 3D GAN and AtlasNet in both quantitative and qualitative measures.
- Quantitative Performance: On human shape data, Rank3DGAN achieved a Normal Consistency of 0.9561 and an Intersection over Union (IoU) of 0.8146, surpassing voxel-based methods.
- Qualitative Control: The model produces smooth interpolations. Adjusting the attribute variable $r$ results in realistic deformations—such as a bird's wings lifting or a body becoming heavier—without surface artifacts or noise.
- Disentanglement: A key strength is the ability to isolate attributes; for instance, changing "age" on a face does not inadvertently alter "gender" or "expression."
Conclusion and Future Work
Rank3DGAN demonstrates that relative comparisons are a powerful signal for learning semantic control in 3D generation. By treating meshes as 2D charts, it maintains geometric detail while utilizing efficient 2D architectures. Current limitations include a restriction to zero-genus topologies (shapes without holes) and resolution constraints of the 2D charts. Future research may explore high-resolution GANs and more complex topological supports.
Summary updated