EAD-GAN: A Generative Adversarial Network for Disentangling Affine Transforms in Images

Letao Liu, Xudong Jiang, Martin Saerbeck, Justin Dauwels

Abstract

Generative Adversarial Networks (GANs) often struggle to achieve interpretable disentanglement of latent factors, particularly geometric transformations. This paper presents EAD-GAN, which integrates explicit affine transformation mathematics into the network architecture. By employing a novel Affine Block and Least-Squares Estimation (LSE), EAD-GAN forces specific latent dimensions to correspond to physical properties like rotation and zoom. Experimental results on benchmark datasets demonstrate that EAD-GAN achieves state-of-the-art disentanglement scores and provides consistent, deterministic control over image generation.

Introduction

In the development of Generative Adversarial Networks (GANs), "disentanglement" remains a primary objective: the ability to manipulate a single latent dimension to control exactly one feature—such as the rotation of a face—without affecting other attributes like expression or identity. Existing state-of-the-art models, such as InfoGAN or $\beta$-VAE, typically attempt to achieve this implicitly through statistical regularizers. However, these methods often lack physical interpretability and suffer from stochastic assignment, where the function of a latent dimension varies across different training runs.

Methodology

EAD-GAN addresses these limitations by introducing an explicit inductive bias based on affine transform mathematics. The core of the architecture is the Affine Block, which hard-codes geometric constraints into the loss function.

The Affine Block and LSE

The model constructs a formal $3 \times 3$ affine matrix $M$ from a subset of the latent code $c$. This matrix explicitly defines parameters for rotation ($\theta$), horizontal/vertical zoom ($p, q$), and translation ($x, y$). During training, the network generates image pairs: a reference sample ($X_r$) and a version transformed by the affine matrix ($X_t$).

The EAD-GAN architecture featuring the Affine Block for explicit geometric control.

To ensure the latent codes accurately map to these transformations, an encoder $E$ predicts the transformation parameters from the image pair. Because the relationship between a matrix and its parameters is nonlinear and overdetermined, the authors utilize Least-Squares Estimation (LSE) to infer parameters accurately, closing the loop between the latent space and the generated geometry.

Experimental Evaluation

The authors evaluated EAD-GAN on four benchmark datasets:

  1. MNIST: Handwritten digits.
  2. CelebA: Large-scale celebrity face images.
  3. dSprites: 2D shapes with known ground-truth factors.
  4. Colored dSprites: Shapes with added RGB color variations to test non-geometric generalization.

Key Results

EAD-GAN significantly outperformed previous state-of-the-art (SOTA) methods on quantitative disentanglement metrics. On the dSprites dataset, it achieved a Mutual Information Gap (MIG) of 0.59 and a DCI score of 0.96, compared to previous SOTA scores of approximately 0.37 and 0.71, respectively.

Qualitative results showing disentangled affine transformations on MNIST digits across different latent dimensions.

Qualitatively, latent traversals demonstrate remarkably clean isolation of traits. As shown in the MNIST results, moving along specific latent dimensions ($c_1$ through $c_5$) results in pure geometric transformations without altering the digit's identity. Furthermore, EAD-GAN provides deterministic mapping, ensuring that a specific latent dimension consistently controls the same physical property across different training trials.

Conclusion and Future Work

By treating affine transforms as known mathematical properties rather than learned statistical ones, EAD-GAN achieves superior modularity and consistency. While the model masters affine transformations, the "non-affine" components of the latent code (e.g., digit thickness) are still learned implicitly. Future research may extend this framework to 3D transformations and more complex expert-knowledge regularizers to disentangle abstract features beyond simple geometry.

Summary updated

Abstract

This article proposes a generative adversarial network called explicit affine disentangled generative adversarial network (EAD-GAN), which explicitly disentangles affine transform in a self-supervised manner. We propose an affine transform regularizer to force the InfoGAN to have explicit properties of affine transform. To facilitate training an affine transform encoder, we decompose the affine matrix into two separate matrices and infer the explicit transform parameters by the least-squares method. Unlike the existing approaches, representations learned by the proposed EAD-GAN have clear physical meaning, where transforms, such as rotation, horizontal and vertical zooms, skews, and translations, are explicitly learned from training data. Thus, we set different values of each transform parameter individually to generate specifically affine transformed data by the learned network. We show that the proposed EAD-GAN successfully disentangles these attributes on the MNIST, CelebA, and dSprites datasets. EAD-GAN achieves higher disentanglement scores with a large margin compared to the state-of-the-art methods on the dSprites dataset. For example, on the dSprites dataset, EAD-GAN achieves the MIG and DCI score of 0.59 and 0.96 respectively, compared to 0.37 and 0.71, respectively, for the state-of-the-art methods.
BibTeX
@article{liu2024ead,
  author = {Letao Liu and Xudong Jiang and Martin Saerbeck and Justin Dauwels},
  title = {{EAD-GAN: A Generative Adversarial Network for Disentangling Affine Transforms in Images}},
  journal = {IEEE Trans. Neural Networks Learn. Syst.},
  year = {2024},
  doi = {10.1109/tnnls.2022.3195533},
}
Figure