C2VAE: Gaussian Copula-based VAE Differing Disentangled from Coupled Representations with Contrastive Posterior

Zhangkai Wu, Longbing CaoarXiv 2023 · DOI · arXiv

Introduction

In the field of Variational Autoencoders (VAEs), achieving disentanglement—the ability to separate independent factors of variation in data (e.g., shape vs. color)—remains a primary objective. While current state-of-the-art methods typically penalize Total Correlation (TC) to enforce latent independence, they often rely on the assumption that hidden features are either perfectly independent or identically distributed. In practice, feature dependencies are heterogeneous; some factors are strongly "coupled," leading to unstable training or suboptimal trade-offs between reconstruction quality and disentanglement.

The Contrastive Copula VAE (C2VAE) addresses this by explicitly modeling feature dependencies using Neural Gaussian Copulas. By distinguishing between "clean" factorized representations and "messy" coupled ones through a self-supervised contrastive classifier, the model effectively filters out entangled factors.

Methodology

The C2VAE architecture extends the standard VAE framework with two primary components designed to handle feature coupling:

  1. Neural Gaussian Copula: Copulas describe the relationship between random variables independently of their marginal distributions. C2VAE employs an auxiliary encoder branch to learn a "neural copula" that captures the correlation structure (the "glue") between hidden features, generating "coupled representations."
  2. Self-supervised Contrastive Classifier: A classifier ($\psi$) is trained to differentiate between standard factorized representations ($z_q$) and the coupled representations ($z_p$) generated by the copula.
  3. Contrastive Loss: By minimizing a contrastive loss ($\mathcal{L}_C$), the VAE is incentivized to "filter out" dependencies identified by the copula, forcing the primary latent space to retain only independent, explanatory features.
The C2VAE architecture featuring the encoder, decoder, and the contrastive classification branch using Neural Gaussian Copulas.

The training process involves a two-phase optimization: updating VAE parameters for reconstruction and independence (via TC loss), and updating the contrastive classifier to better distinguish clean from coupled factors.

Experimental Setup

C2VAE was evaluated on four standard disentanglement benchmarks:

  • dSprites: 2D shapes (heart, oval, square).
  • SmallNORB: 3D toy objects under varying conditions.
  • 3D Shapes: Objects with varying floor/wall colors and scales.
  • 3D Cars: 3D car models at different rotation angles.

Results and Discussion

C2VAE consistently outperformed baselines such as $\beta$-VAE, $\beta$-TCVAE, and FactorVAE. Key performance indicators included:

  • Higher Disentanglement Scores: Superior SAP (Separated Attribute Predictability) and FAC (FactorVAE score) results. On dSprites, C2VAE achieved an SAP of 0.044, significantly higher than FactorVAE’s 0.028.
  • Improved Factorization: The model achieved the lowest WCN (Normalized Wasserstein Distance), indicating a posterior distribution closer to perfect factorization.
  • Training Stability: Unlike many TC-based VAEs that are sensitive to initialization, C2VAE demonstrates a smoother and more consistent learning curve.
Comparison of training stability between C2VAE and FactorVAE, showing lower variance and smoother convergence for C2VAE.

Qualitatively, "latent traversal" visualizations showed that C2VAE is more effective at isolating difficult factors, such as "orientation" in dSprites, which frequently entangles with "shape" in competing models.

Conclusion

By moving beyond the standard IID assumption and explicitly modeling dependencies via Neural Gaussian Copulas, C2VAE provides a more robust framework for unsupervised disentanglement. The contrastive mechanism not only improves the quality of learned representations but also stabilizes the training process. Future work may explore more complex copula structures, such as Student-t or Mixture Copulas, for diverse data types like time-series or text.

Summary updated

Abstract

We present a self-supervised variational autoencoder (VAE) to jointly learn disentangled and dependent hidden factors and then enhance disentangled representation learning by a self-supervised classifier to eliminate coupled representations in a contrastive manner. To this end, a Contrastive Copula VAE (C$^2$VAE) is introduced without relying on prior knowledge about data in the probabilistic principle and involving strong modeling assumptions on the posterior in the neural architecture. C$^2$VAE simultaneously factorizes the posterior (evidence lower bound, ELBO) with total correlation (TC)-driven decomposition for learning factorized disentangled representations and extracts the dependencies between hidden features by a neural Gaussian copula for copula coupled representations. Then, a self-supervised contrastive classifier differentiates the disentangled representations from the coupled representations, where a contrastive loss regularizes this contrastive classification together with the TC loss for eliminating entangled factors and strengthening disentangled representations. C$^2$VAE demonstrates a strong effect in enhancing disentangled representation learning. C$^2$VAE further contributes to improved optimization addressing the TC-based VAE instability and the trade-off between reconstruction and representation.
BibTeX
@misc{wu2023c,
  author = {Zhangkai Wu and Longbing Cao},
  title = {{C2VAE: Gaussian Copula-based VAE Differing Disentangled from Coupled Representations with Contrastive Posterior}},
  year = {2023},
  eprint = {2309.13303},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.13303},
}
Figure