Abstract
One noted issue of vector-quantized variational autoencoder (VQ-VAE) is that the learned discrete representation uses only a fraction of the full capacity of the codebook, also known as codebook collapse. We hypothesize that the training scheme of VQ-VAE, which involves some carefully designed heuristics, underlies this issue. In this paper, we propose a new training scheme that extends the standard VAE via novel stochastic dequantization and quantization, called stochastically quantized variational autoencoder (SQ-VAE). In SQ-VAE, we observe a trend that the quantization is stochastic at the initial stage of the training but gradually converges toward a deterministic quantization, which we call self-annealing. Our experiments show that SQ-VAE improves codebook utilization without using common heuristics. Furthermore, we empirically show that SQ-VAE is superior to VAE and VQ-VAE in vision- and speech-related tasks.
Introduction
Vector Quantized Variational Autoencoders (VQ-VAEs) have become a cornerstone of modern generative modeling, particularly for high-fidelity image synthesis. However, they suffer from a persistent challenge: codebook collapse. In this state, the model utilizes only a small subset of its available discrete codes, wasting capacity and degrading reconstruction quality. To mitigate this, practitioners have historically relied on non-Bayesian heuristics such as the straight-through estimator, stop-gradient operators, and exponential moving averages (EMA). While these "tricks" enable training, they depart from the theoretically grounded Variational Bayes framework, making models difficult to tune and mathematically inconsistent.
Methodology: Stochastic Quantization
The proposed SQ-VAE framework addresses codebook collapse by re-integrating discrete representation learning into the standard Variational Bayes objective. The core innovation lies in replacing deterministic nearest-neighbor lookup with stochastic dequantization and quantization.
In SQ-VAE, the process is probabilistic rather than deterministic:
- Stochastic Encoding: Inputs are mapped to a distribution in the latent space rather than a single point.
- Stochastic Quantization: Instead of a hard assignment to the nearest codebook vector, a discrete code is sampled based on a probability distribution.
- Self-Annealing: A critical emergent property of SQ-VAE is self-annealing. During optimization of the Evidence Lower Bound (ELBO), the variance of the latent distributions naturally decreases. The model begins with high stochasticity—allowing it to explore the entire codebook—and gradually converges to a deterministic state as training progresses. This schedule is learned automatically through backpropagation, eliminating the need for manual hyperparameter tuning.
The authors introduce two variants: Gaussian SQ-VAE for continuous data (e.g., standard images) and vMF SQ-VAE (using the von Mises-Fisher distribution) for categorical data like segmentation masks.
Key Insights and Empirical Results
The shift from deterministic to stochastic quantization provides several advantages. By making the quantization process probabilistic, the model is forced to utilize the full codebook during the early stages of training.
Codebook Utilization and Reconstruction
SQ-VAE significantly outperforms VQ-VAE in codebook utilization, measured by perplexity. While VQ-VAE often uses only a fraction of its capacity, SQ-VAE frequently utilizes nearly the entire codebook. This improved utilization translates directly to better reconstruction:
- Vision: On the CelebA dataset, SQ-VAE achieved a Mean Squared Error (MSE) of 0.96, compared to 1.33 for VQ-VAE with EMA.
- Speech: In log-Mel spectrogram reconstruction (VCTK dataset), SQ-VAE reached an MSE of 24.68 dB², surpassing the tuned VQ-VAE baseline of 29.59 dB².
Generation Quality
When paired with a PixelCNN prior for image generation, SQ-VAE produces higher-quality samples. On CelebA, it achieved a Fréchet Inception Distance (FID) of 28.2, a substantial improvement over the VQ-VAE baseline of 42.0.
Conclusion
SQ-VAE offers a principled, theoretically sound alternative to the heuristic-driven training of VQ-VAEs. By introducing stochasticity that self-anneals, it effectively solves codebook collapse and achieves superior performance across vision and speech tasks. This work demonstrates that returning to Bayesian fundamentals can simplify model training while simultaneously improving results.
Summary updated