Abstract
Generative models that learn disentangled representations for different factors of variation in an image are crucial for targeted data augmentation. We introduce a novel architecture that disentangles the latent space into two complementary subspaces using only weak supervision via pairwise similarity labels. By integrating cycle-consistency into a variational auto-encoder (VAE) framework, we avoid the instabilities of adversarial training. Our results across three datasets demonstrate superior disentanglement and robustness compared to state-of-the-art adversarial methods.
1. Introduction
Disentangling factors of variation involves training a model to decompose an image into independent components, such as identity (e.g., digit class) and style (e.g., slant). Achieving this allows for precise data manipulation and augmentation. However, learning these representations is difficult when labels are sparse or when models fall into "degenerate" states.
2. The Degeneracy Trap
A common failure mode in auto-encoders is the "degeneracy trap," where the model ignores provided labels and encodes all information into the unspecified latent subspace.
To combat this, previous works often relied on Generative Adversarial Networks (GANs). While effective, GANs are notoriously difficult to train, prone to mode collapse, and highly sensitive to hyperparameter choices.
3. Methodology: Cycle-Consistent VAEs
We propose a non-adversarial framework that splits the latent space into a specified subspace ($s$) and an unspecified subspace ($z$). The core innovation is the application of cycle-consistency in two directions:
- Forward Cycle: Given two images of the same class, the model swaps their $s$ vectors and attempts to reconstruct the original images, ensuring $s$ captures shared identity.
- Reverse Cycle: To prevent identity leakage into $z$, we sample a random $z$ and combine it with two different identity vectors ($s_1, s_2$) to generate new images. The encoder must then extract the same $z$ from both generated images, forcing the $z$ space to be independent of identity.
4. Experimental Results
The model was evaluated on MNIST, 2D Sprites, and LineMod datasets. A key finding is the model's robustness to the dimensionality of the latent space compared to non-adversarial baselines.
Quantitatively, our model showed significantly lower identity leakage in the $z$-space. On MNIST, identity classification accuracy from the $z$-space was reduced to ~17-26%, compared to ~66-98% in competing models. This indicates that the $z$ space contains far less "leaked" identity information.
5. Conclusion
Cycle-Consistent VAEs provide a stable, non-adversarial alternative for learning disentangled representations. By focusing on encoder-centric cycle-consistency, we achieve robust separation of factors with only weak supervision. Future work will explore improving image sharpness and extending the framework to fully unsupervised settings.
Summary updated