Introduction
Disentangled Representation Learning (DRL) aims to develop models that represent independent factors of variation in data through separate, interpretable latent variables. For instance, in a generative model of digits, one might want separate "knobs" for the digit's identity, its rotation, and its thickness. While unsupervised methods like the $\beta$-VAE have made significant strides, theoretical work suggests that true disentanglement is often impossible without inductive biases—prior knowledge about the structure of the physical world.
In this work, Detlefsen and Hauberg (NeurIPS 2019) introduce the Variational Inferred Transformational Autoencoder (VITAE). VITAE explicitly separates an object's intrinsic appearance (the "what") from its spatial perspective (the "where" and "how") by hard-wiring geometric transformations into the model architecture.
The VITAE Architecture
The core innovation of VITAE is the decomposition of the latent space into two distinct components:
- Perspective ($z_P$): This space encodes spatial transformations such as rotation, translation, and scaling.
- Appearance ($z_A$): This space encodes the "archetype" or canonical form of the object, effectively the image after all spatial transformations have been removed.
To achieve robust separation, the authors proposed Conditional-VITAE (C-VITAE). In this variant, the appearance encoder is conditioned on the predicted perspective. By "undoing" the transformation before encoding the appearance, the model ensures that $z_A$ only needs to represent a normalized version of the data, preventing the appearance space from being polluted by pose information.
Diffeomorphic Transformations
A critical requirement for VITAE is that the spatial transformations must be smooth and invertible (diffeomorphic). This ensures that the model can map pixels back and forth between the observed image and the canonical archetype without losing information. The authors employ:
- Affine transformations via matrix exponentials for simpler datasets like MNIST.
- CPAB (Continuously Piecewise-Affine Based) transformations for complex, non-rigid deformations, such as those found in human body poses and facial expressions.
Experimental Results
The model was evaluated on several benchmarks, including dSprites, Augmented MNIST, and the SMPL human body model.
MNIST Disentanglement
On the Augmented MNIST dataset, C-VITAE successfully separates digit identity from orientation and position. As shown in the latent space visualizations, the perspective space captures the variations in pose (slant and rotation), while the appearance space clusters digits by their canonical forms.
Human Body Modeling (SMPL)
The most striking results were observed on the SMPL dataset, which consists of synthetic 3D human bodies. C-VITAE achieved a disentanglement score ($D_{score}$) of 0.943, significantly outperforming the $\beta$-VAE baseline (0.525). The model demonstrated the ability to independently manipulate body shape (e.g., body fat percentage) and body pose (e.g., arm positions), a task where standard VAEs often fail due to the inherent entanglement of these factors in pixel space.
Conclusion
VITAE represents a shift from purely unsupervised disentanglement towards models with "hard-wired" geometric priors. By incorporating the concept of spatial transformations directly into the architecture, the model achieves superior performance on complex data. Future work may extend this hierarchical approach to disentangle other factors such as color, texture, and lighting, peeling off one layer of variation at a time to reach a fully interpretable representation of the world.
Summary updated