Abstract
The Disentanglement-PyTorch library is a modular, open-source framework designed to streamline the training and evaluation of variational autoencoders (VAEs) for representation learning. By decoupling neural architectures from training algorithms, it enables researchers to systematically explore various loss functions and configurations. The library's utility was demonstrated by the authors' 3rd place finish in the NeurIPS 2019 Disentanglement Challenge.
Introduction
In representation learning, the goal of "disentanglement" is to ensure that individual units in a latent space correspond to specific physical attributes of the data, such as object color or camera angle. Ideally, modifying a single latent variable should result in a change to only one feature in the generated output.
However, achieving unsupervised disentanglement is challenging due to the fragmentation of research, where inconsistent architectures, datasets, and metrics often obscure the true performance of new algorithms. Disentanglement-PyTorch addresses this by providing a unified, modular environment that isolates these variables, ensuring experiments are both reproducible and comparable.
Methods
The primary innovation of Disentanglement-PyTorch is its modular architecture, which allows for high flexibility in experimental design:
- Decoupled Components: Neural network architectures, latent space dimensions, and training algorithms are treated as independent modules. This allows for swapping encoders or decoders without altering the core VAE logic.
- Plug-and-Play Loss Terms: Objective functions, such as the Total Correlation term in $\beta$-TCVAE, are implemented as plug-ins. Researchers can combine different loss terms via command-line flags to study their interactions.
- Advanced Scheduling: The library supports techniques like "Controlled Capacity Increase," which gradually relaxes the information bottleneck to improve disentanglement while maintaining reconstruction quality.
- Standardized Evaluation: Integration with Google’s
disentanglement-libprovides access to standard metrics like the Mutual Information Gap (MIG) and FactorVAE score.
The framework supports a variety of unsupervised algorithms, including $\beta$-VAE, Factor-VAE, DIP-VAE, and $\beta$-TCVAE, as well as conditional approaches like CVAE.
Datasets
The library was evaluated using the MPI3D dataset suite from the NeurIPS 2019 challenge, which includes:
- MPI3D-toy: Synthetic images of a robotic arm.
- MPI3D-realistic: Higher-fidelity synthetic renders.
- MPI3D-real: Physical photographs of the robotic setup.
Key Results
The library's effectiveness was validated by the authors' 3rd place ranking in the NeurIPS 2019 Disentanglement Challenge. Quantitative results using the $\beta$-TCVAE algorithm on the mpi3d_real dataset showed a DCI score of 0.4044 and an IRS of 0.6423, as detailed in the performance table below.
Qualitative success is evident in latent factor traversals, which show the model successfully isolating features such as "Object Shape" and "Horizontal Axis." However, the authors noted instances of partial entanglement, particularly between camera height and background color.
Discussion and Future Work
While Disentanglement-PyTorch provides a robust toolset, the authors acknowledge that achieving perfect disentanglement remains a difficult task. Their results highlighted "partial entanglement," where changes in one latent factor subtly influenced others.
Future development will focus on integrating more complex architectures, such as auto-regressive models, and expanding support to broader data domains beyond 3D shapes and robotic arms. The library is open-source and available for the research community on GitHub.
Summary updated