Introduction
Deep learning models, particularly Generative Adversarial Networks (GANs) and Autoencoders, typically learn "latent representations"—compact numerical codes representing data. However, these codes are often entangled, meaning a single value might influence multiple semantic features simultaneously (e.g., age, gender, and pose). While disentangled representations are highly desirable for interpretability and control, most existing methods require training models from scratch. This paper introduces the Factors Decomposer-Entangler Network (FDEN), a "plug-in" module designed to factorize existing, entangled latent spaces into independent, interpretable components without modifying the original model's weights.
The FDEN Framework
The FDEN architecture functions as an autoencoder for latent vectors, comprising three primary modules:
- Decomposer ($\mathcal{D}$): This module maps the original latent vector $z$ into a set of discrete factors $f_0, f_1, ..., f_N$.
- Factorizer ($\mathcal{F}$): Serving as the core of the disentanglement process, the Factorizer employs a Statisticians Network to minimize Total Correlation (a multi-variable extension of mutual information) between factors, ensuring statistical independence. An Alignment Network provides semi-supervised guidance, linking specific factors to human-understandable attributes using minimal labeled data.
- Entangler ($\mathcal{E}$): This module recombines the factors to reconstruct the original latent vector $\tilde{z}$.
To ensure stable training, the authors utilize the Donsker-Varadhan (DV) representation for mutual information estimation and a Gradient Reversal Layer (GRL) for effective minimization. Adaptive Gradient Clipping is further employed to manage potentially unbounded mutual information gradients.
Experimental Evaluation
The efficacy of FDEN was validated across several diverse domains:
- Faces: MS-Celeb-1M and CelebA (40 binary attributes).
- Characters: Omniglot handwritten characters.
- Natural Images: Mini-ImageNet.
- Flowers: Oxford Flower dataset.
- Synthetic Shapes: dSprites benchmark.
Results and Discussion
FDEN demonstrates significant capabilities in semantic image manipulation and few-shot learning. By replacing specific factors while maintaining others (e.g., keeping "identity" constant while changing "style"), the model enables precise attribute control across various image types.
Quantitatively, FDEN achieved state-of-the-art performance on the dSprites benchmark with a Mutual Information Gap (MIG) of 0.5992, surpassing established models like $\beta$-VAE and FactorVAE. In few-shot classification on Omniglot, FDEN reached 95.4% accuracy (5-shot), significantly outperforming standard MLP baselines.
Conclusion and Limitations
The "plug-in" nature of FDEN allows for the enhancement of pre-trained models without retraining. However, its performance is bounded by the quality of the underlying model; features ignored during initial training cannot be recovered. Future work will explore more robust dependency measures, such as Wasserstein distance, to further refine factor independence.
Summary updated