Abstract
Variational autoencoders (VAEs) with an auto-regressive decoder have been applied for many natural language processing (NLP) tasks. The VAE objective consists of two terms, (i) reconstruction and (ii) KL regularization, balanced by a weighting hyper-parameter $\beta$. One notorious training difficulty is that the KL term tends to vanish. In this paper we study scheduling schemes for $\beta$, and show that KL vanishing is caused by the lack of good latent codes in training the decoder at the beginning of optimization. To remedy this, we propose a cyclical annealing schedule, which repeats the process of increasing $\beta$ multiple times. This new procedure allows the progressive learning of more meaningful latent codes, by leveraging the informative representations of previous cycles as warm re-starts. The effectiveness of cyclical annealing is validated on a broad range of NLP tasks, including language modeling, dialog response generation and unsupervised language pre-training.
Introduction: The "Lazy" Decoder and KL Vanishing
Variational Autoencoders (VAEs) are a staple of generative modeling, designed to map high-dimensional data into a structured latent space. In Natural Language Processing (NLP), VAEs typically employ an autoregressive decoder (e.g., an LSTM or Transformer) to generate text. However, these models frequently suffer from KL vanishing, where the KL divergence term in the objective function drops to zero during training.
This phenomenon can be understood through a "Two-Path Theory." An autoregressive decoder has two sources of information to predict the next token:
- Path A (Latent Path): The global latent code $z$ provided by the encoder.
- Path B (Autoregressive Path): The local history of previously generated tokens.
Because Path B is significantly easier to optimize early in training, the model often learns to ignore the latent code entirely. Consequently, the encoder fails to learn meaningful representations, and the VAE collapses into a standard language model, losing the benefits of a structured latent space.
The Cyclical Annealing Schedule
The standard approach to mitigate this is monotonic annealing, where the weight of the KL term ($\beta$) is gradually increased from 0 to 1. While this provides a window for the model to utilize the latent path, it often fails to prevent collapse if the initial learning is insufficient.
We propose a Cyclical Annealing Schedule. Instead of a single ramp-up, the training process is divided into $M$ cycles. Each cycle consists of:
- An Annealing Stage: $\beta$ increases from 0 to 1, encouraging the model to explore and incorporate latent information.
- A Fixing Stage: $\beta$ is held at 1, allowing the model to optimize the full VAE objective and stabilize.
By repeating this process, the model uses the informative representations learned in the previous cycle as a "warm restart" for the next. This allows for the progressive refinement of the latent space, effectively "dislodging" the model from poor local optima where the latent code is ignored.
Experimental Results
The cyclical annealing schedule was evaluated across three key NLP domains: language modeling (PTB), dialog generation (Switchboard), and unsupervised pre-training (Yelp).
Latent Space Evolution
A key advantage of the cyclical approach is its ability to progressively structure the latent space. As shown in the visualization below, while a constant or monotonic schedule may lead to a cluttered or collapsed latent space, the cyclical schedule allows the model to learn increasingly distinct and meaningful clusters over successive cycles.
Performance Gains
- Language Modeling: On the PTB dataset, cyclical annealing achieved higher KL values and lower reconstruction error, indicating better utilization of the latent space.
- Dialog Diversity: In dialog tasks, models trained with cyclical annealing produced more diverse and contextually relevant responses, avoiding the generic "safe" responses (e.g., "I don't know") common in collapsed VAEs.
- Downstream Utility: Latent codes generated via cyclical annealing proved superior for unsupervised pre-training, significantly improving classification accuracy on the Yelp dataset when labeled data was limited.
Conclusion
Cyclical annealing offers a simple, zero-cost modification to the VAE training pipeline that effectively mitigates KL vanishing. By periodically re-opening the latent path, the model is encouraged to learn more expressive representations. This approach not only improves generative performance but also yields more useful latent features for downstream tasks. Future work may explore adaptive scheduling techniques that adjust cycle parameters based on real-time training dynamics.
Summary updated