Variational Learning with Disentanglement-PyTorch

Amir H. Abdi, Purang Abolmaesumi, Sidney S. FelsarXiv 2019 · arXiv

Abstract

The Disentanglement-PyTorch library is a modular, open-source framework designed to streamline the training and evaluation of variational autoencoders (VAEs) for representation learning. By decoupling neural architectures from training algorithms, it enables researchers to systematically explore various loss functions and configurations. The library's utility was demonstrated by the authors' 3rd place finish in the NeurIPS 2019 Disentanglement Challenge.

Introduction

In representation learning, the goal of "disentanglement" is to ensure that individual units in a latent space correspond to specific physical attributes of the data, such as object color or camera angle. Ideally, modifying a single latent variable should result in a change to only one feature in the generated output.

However, achieving unsupervised disentanglement is challenging due to the fragmentation of research, where inconsistent architectures, datasets, and metrics often obscure the true performance of new algorithms. Disentanglement-PyTorch addresses this by providing a unified, modular environment that isolates these variables, ensuring experiments are both reproducible and comparable.

Methods

The primary innovation of Disentanglement-PyTorch is its modular architecture, which allows for high flexibility in experimental design:

  • Decoupled Components: Neural network architectures, latent space dimensions, and training algorithms are treated as independent modules. This allows for swapping encoders or decoders without altering the core VAE logic.
  • Plug-and-Play Loss Terms: Objective functions, such as the Total Correlation term in $\beta$-TCVAE, are implemented as plug-ins. Researchers can combine different loss terms via command-line flags to study their interactions.
  • Advanced Scheduling: The library supports techniques like "Controlled Capacity Increase," which gradually relaxes the information bottleneck to improve disentanglement while maintaining reconstruction quality.
  • Standardized Evaluation: Integration with Google’s disentanglement-lib provides access to standard metrics like the Mutual Information Gap (MIG) and FactorVAE score.

The framework supports a variety of unsupervised algorithms, including $\beta$-VAE, Factor-VAE, DIP-VAE, and $\beta$-TCVAE, as well as conditional approaches like CVAE.

Datasets

The library was evaluated using the MPI3D dataset suite from the NeurIPS 2019 challenge, which includes:

  • MPI3D-toy: Synthetic images of a robotic arm.
  • MPI3D-realistic: Higher-fidelity synthetic renders.
  • MPI3D-real: Physical photographs of the robotic setup.

Key Results

The library's effectiveness was validated by the authors' 3rd place ranking in the NeurIPS 2019 Disentanglement Challenge. Quantitative results using the $\beta$-TCVAE algorithm on the mpi3d_real dataset showed a DCI score of 0.4044 and an IRS of 0.6423, as detailed in the performance table below.

Quantitative performance of the beta-TCVAE algorithm across different versions of the MPI3D dataset.

Qualitative success is evident in latent factor traversals, which show the model successfully isolating features such as "Object Shape" and "Horizontal Axis." However, the authors noted instances of partial entanglement, particularly between camera height and background color.

Latent factor traversals demonstrating the model's ability to isolate specific physical attributes in the MPI3D dataset.

Discussion and Future Work

While Disentanglement-PyTorch provides a robust toolset, the authors acknowledge that achieving perfect disentanglement remains a difficult task. Their results highlighted "partial entanglement," where changes in one latent factor subtly influenced others.

Future development will focus on integrating more complex architectures, such as auto-regressive models, and expanding support to broader data domains beyond 3D shapes and robotic arms. The library is open-source and available for the research community on GitHub.

Summary updated

Abstract

Unsupervised learning of disentangled representations is an open problem in machine learning. The Disentanglement-PyTorch library is developed to facilitate research, implementation, and testing of new variational algorithms. In this modular library, neural architectures, dimensionality of the latent space, and the training algorithms are fully decoupled, allowing for independent and consistent experiments across variational methods. The library handles the training scheduling, logging, and visualizations of reconstructions and latent space traversals. It also evaluates the encodings based on various disentanglement metrics. The library, so far, includes implementations of the following unsupervised algorithms VAE, Beta-VAE, Factor-VAE, DIP-I-VAE, DIP-II-VAE, Info-VAE, and Beta-TCVAE, as well as conditional approaches such as CVAE and IFCVAE. The library is compatible with the Disentanglement Challenge of NeurIPS 2019, hosted on AICrowd, and achieved the 3rd rank in both the first and second stages of the challenge.
BibTeX
@misc{abdi2019variational,
  author = {Amir H. Abdi and Purang Abolmaesumi and Sidney S. Fels},
  title = {{Variational Learning with Disentanglement-PyTorch}},
  year = {2019},
  eprint = {1912.05184},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1912.05184},
}
Figure