$ζ$-QVAE: A Quantum Variational Autoencoder utilizing Regularized Mixed-state Latent Representations

Gaoyuan Wang, Jonathan Warrell, Prashant S. Emani, Mark GersteinarXiv 2024 · DOI · arXiv

TL;DR

The $\zeta$-QVAE is a new, fully quantum variational autoencoder that uses "mixed states" to compress large datasets into compact quantum representations. By introducing a regularized latent space—much like its classical VAE cousins—it outperforms previous quantum models in data generation and downstream classification tasks while using significantly fewer parameters than classical models.

What Problem Does This Paper Address?

Quantum computers are currently in the "Noisy Intermediate-Scale Quantum" (NISQ) era, meaning we have very few qubits to work with. This makes processing large, real-world datasets (like genomic data) incredibly difficult. To make these datasets "quantum-ready," we need to compress them into low-dimensional representations without losing essential information.

While classical machine learning has the Variational Autoencoder (VAE) to handle compression and data generation, the quantum world has struggled to keep up. Existing "Quantum Autoencoders" (QAEs) usually lack regularization—the mathematical "nudge" that ensures the compressed space is well-structured for generating new data or generalizing to unseen examples. Furthermore, many current models are "hybrid," meaning they can't be trained directly on pure quantum data. The authors of this paper sought to bridge this gap with a framework that is quantum from top to bottom.

Methods

The authors introduce the $\zeta$-QVAE (where $\zeta$ represents the density matrix of the latent state). Unlike standard quantum circuits that deal only with "pure" states, this model embraces mixed states (probabilistic mixtures of quantum states) at every stage.

Key technical highlights include:

  • The Latent Prior: Just as a classical VAE tries to map data to a standard Gaussian distribution, the $\zeta$-QVAE uses the maximally mixed state as its "prior." This encourages the model to use the full capacity of the available quantum space.
  • Flexible Loss Functions: The framework isn't locked into one metric. It supports various quantum "distances," including Fidelity, Quantum Relative Entropy (KLD), Jensen-Shannon Divergence (JSD), and even the Quantum Wasserstein distance.
  • Auxiliary Qubits: The model uses extra "trash" and "auxiliary" qubits. This allows for non-unitary transformations, effectively "decoupling" the act of compressing data from the act of regularizing it—a feature that has no direct classical analog.
  • Global vs. Instance Training: The model can be trained on individual data points or on a "global density matrix" representing the entire dataset at once.

Datasets

The researchers evaluated $\zeta$-QVAE on three distinct types of data:

  1. Synthetic Quantum Dataset: 1,000 two-qubit states designed to have a structure with no classical analog, testing the model's ability to handle "intrinsically quantum" information.
  2. PsychENCODE Genomics Data: Real-world gene expression data from ~1,500 subjects (schizophrenia patients and controls). This classical data was mapped into quantum states using amplitude encoding.
  3. Swiss Roll Dataset: A classic machine learning benchmark involving a 3D "spiral" sheet of data, used to test how well the model captures non-linear manifolds.

Key Insights and Ideas

  • The Power of Entanglement: The authors suggest that the entanglement between qubits allows the model to capture intricate dependencies in data more efficiently than classical bits, leading to better compression with fewer parameters.
  • Decoupling via Auxiliaries: One of the most interesting findings is that adding auxiliary qubits allows the model to optimize reconstruction and regularization separately. In models without these extra qubits, these two goals are often in conflict.
  • Privacy-Preserving Potential: Because the "Global" objective only requires the aggregate density matrix of a dataset, it could theoretically allow for training on sensitive data (like medical records) without ever needing to access individual data points, offering a new path for federated learning.
  • Parameter Efficiency: A 3-layer $\zeta$-QVAE used only 60 parameters to process 16-feature genomic data, whereas a comparable classical linear VAE required 216 parameters.

Key Results

The $\zeta$-QVAE proved to be a powerhouse in both reconstruction and classification:

  • Genomics Performance: On the PsychENCODE dataset, the $\zeta$-QVAE (using 3 latent qubits) achieved a classification accuracy of 0.669, nearly matching the performance of the full, uncompressed dataset (0.675) and outperforming classical VAE-based approaches.
  • Swiss Roll Success: The model achieved a classification accuracy of 0.77, which was slightly higher than the best-matched classical $\beta$-VAE.
  • Latent Space Utilization: By tuning the regularization parameter ($\beta$), the researchers showed they could significantly increase the "volume" of the latent space being used, which directly correlated with better performance on downstream tasks.

Limitations and Future Work

Despite the promising results, the authors acknowledge several hurdles for near-term deployment:

  • Hardware Constraints: Implementing this on actual NISQ hardware is challenging due to the need for high-accuracy state tomography (reading out the quantum states) and efficient state preparation.
  • Barren Plateaus: Like many quantum neural networks, the model is susceptible to "barren plateaus"—regions in the optimization landscape where gradients vanish, making training difficult as the number of qubits increases.
  • Scaling: While efficient in parameters, the model still needs to be tested on much higher-dimensional data to see if the quantum advantage holds as we move toward "Big Data" scales.

Future work will likely focus on developing "problem-specific" circuit designs (Ansatzes) to avoid training issues and exploring the privacy guarantees of the global training objective.

Summary updated

Abstract

A major challenge in quantum computing is its application to large real-world datasets due to scarce quantum hardware resources. One approach to enabling tractable quantum models for such datasets involves finding low-dimensional representations that preserve essential information for downstream analysis. In classical machine learning, variational autoencoders (VAEs) facilitate efficient data compression, representation learning for subsequent tasks, and novel data generation. However, no quantum model has been proposed that captures these features for direct application to quantum data on quantum computers. Some existing quantum models for data compression lack regularization of latent representations. Others are hybrid models with only some internal quantum components, impeding direct training on quantum data. To address this, we present a fully quantum framework, $ζ$-QVAE, which encompasses all the capabilities of classical VAEs and can be directly applied to map both classical and quantum data to a lower-dimensional space, while effectively reconstructing much of the original state from it. Our model utilizes regularized mixed states to attain optimal latent representations. It accommodates various divergences for reconstruction and regularization. Furthermore, by accommodating mixed states at every stage, it can utilize the full-data density matrix and allow for a training objective defined on probabilistic mixtures of input data. Doing so, in turn, makes efficient optimization possible and has potential implications for private and federated learning. In addition to exploring the theoretical properties of $ζ$-QVAE, we demonstrate its performance on genomics and synthetic data. Our results indicate that $ζ$-QVAE learns representations that better utilize the capacity of the latent space and exhibits similar or better performance compared to matched classical models.
BibTeX
@misc{wang2024qvae,
  author = {Gaoyuan Wang and Jonathan Warrell and Prashant S. Emani and Mark Gerstein},
  title = {{$ζ$-QVAE: A Quantum Variational Autoencoder utilizing Regularized Mixed-state Latent Representations}},
  year = {2024},
  eprint = {2402.17749},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.17749},
}
Figure