SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantization

Yuhta Takida, Takashi Shibuya, Wei-Hsiang Liao, Chieh-Hsin Lai, Junki Ohmura, Toshimitsu Uesaka, Naoki Murata, Shusuke Takahashi, Toshiyuki Kumakura, Yuki Mitsufuji

Introduction

Discrete representation learning, popularized by the Vector-Quantized Variational Autoencoder (VQ-VAE), is a powerful technique for generative modeling because it allows models to capture high-level, categorical concepts. However, VQ-VAE has a well-known limitation: it does not strictly adhere to the standard Variational Bayes framework.

Because the quantization step—mapping a continuous vector to the nearest discrete codebook entry—is non-differentiable, VQ-VAE relies on heuristics like the "straight-through estimator" and "stop-gradient" operators during training. A major consequence of this deterministic approach is codebook collapse, where the model only utilizes a small fraction of the available codebook entries. To mitigate this, researchers often resort to additional heuristics, such as exponential moving averages (EMA) for codebook updates or manual codebook resets.

SQ-VAE (Stochastically Quantized Variational Autoencoder) introduces a principled framework that replaces these heuristic-heavy training methods. By introducing stochastic quantization with trainable parameters, the model exhibits a "self-annealing" effect—naturally transitioning from stochastic to deterministic quantization—which effectively eliminates codebook collapse and improves reconstruction quality across vision and speech tasks.

The SQ-VAE Framework

The core innovation of SQ-VAE is the introduction of a pair of stochastic processes in the latent space: stochastic dequantization and stochastic quantization.

Instead of a hard, deterministic mapping, the model treats quantization as a probability distribution. This allows the entire system to be trained by maximizing the Evidence Lower Bound (ELBO), aligning it perfectly with standard VAE training.

The encoding path of SQ-VAE involves deterministic encoding (E1), stochastic dequantization (E2), and stochastic quantization (E3).

The authors propose two main variants of the architecture to handle different data types:

  1. Gaussian SQ-VAE: Designed for continuous data. The dequantization process is modeled as a Gaussian distribution with a trainable variance ($\Sigma_\phi$).
  2. vMF SQ-VAE: Specialized for categorical data (such as segmentation maps). It uses the von Mises-Fisher (vMF) distribution, which operates on a hypersphere. This is crucial because standard Gaussian losses often perform poorly on categorical cross-entropy tasks.

The Power of Self-Annealing

A defining feature of SQ-VAE is its self-annealing property. As the model optimizes the ELBO, the trainable variance (in the Gaussian variant) or the concentration parameter (in the vMF variant) naturally decreases over time.

This means the quantization starts very "fuzzy" or stochastic, allowing the model to broadly explore many codebook entries early in training. As training converges, the distribution gradually becomes "sharp" or deterministic. Unlike previous methods that required manual, carefully tuned schedules for annealing stochasticity, SQ-VAE's annealing is an emergent property of the optimization process itself.

Empirical evidence of self-annealing: as iterations increase, the entropy of the quantization process naturally decreases for the trainable model compared to fixed baselines.

Key Results and Scalability

The authors evaluated SQ-VAE across a wide range of benchmarks, including continuous vision tasks (MNIST, CIFAR-10, CelebA), categorical vision tasks (CelebAHQ-Mask), and speech tasks (VCTK, ZeroSpeech 2019).

  • Better Reconstruction: On CelebA, Gaussian SQ-VAE achieved a Mean Squared Error (MSE) of $0.96 \times 10^{-3}$, significantly outperforming VQ-VAE trained with EMA ($1.33 \times 10^{-3}$).
  • Higher Codebook Utilization: SQ-VAE consistently maintained higher codebook perplexity (a measure of how many entries are actively used) compared to VQ-VAE.
  • Scalability: One of the most striking findings is that SQ-VAE's performance improves consistently as the codebook size increases. In contrast, standard VQ-VAE often hits a plateau or degrades because it cannot effectively utilize larger codebooks without extensive tuning.
  • Categorical Superiority: On the CelebAHQ-Mask task, vMF SQ-VAE achieved a mean Intersection over Union (mIoU) of 74.6, compared to just 59.7 for VQ-VAE.
  • Speech Performance: SQ-VAE achieved lower MSE in log-Mel spectrogram reconstruction on the VCTK dataset ($24.68 \text{ dB}^2$ vs. $29.59 \text{ dB}^2$ for VQ-VAE).

Limitations and Future Work

While SQ-VAE shows clear advantages in reconstruction quality and codebook utilization, the authors noted that it did not provide a significant boost in certain downstream semantic tasks, such as "acoustic unit discovery" in speech. This suggests that while the reconstruction is mathematically superior, the semantic quality of the learned discrete units might require further investigation or different inductive biases.

Additionally, while SQ-VAE simplifies training by removing heuristics, it introduces a new design choice: choosing a parameterization for the variance (e.g., whether the variance is global, per-sample, or per-dimension). Exploring the optimal parameterizations for different neural architectures remains a promising direction for future research.

Summary updated

Abstract

One noted issue of vector-quantized variational autoencoder (VQ-VAE) is that the learned discrete representation uses only a fraction of the full capacity of the codebook, also known as codebook co...
BibTeX
@inproceedings{takida2022sq,
  author = {Yuhta Takida and Takashi Shibuya and Wei-Hsiang Liao and Chieh-Hsin Lai and Junki Ohmura and Toshimitsu Uesaka and Naoki Murata and Shusuke Takahashi and Toshiyuki Kumakura and Yuki Mitsufuji},
  title = {{SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantization}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=Qk5tFh56iEf},
}
Figure