Towards Text Generation with Adversarially Learned Neural Outlines

Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron C. Courville, Chris PalNeurIPS 2018 · OpenReview

TL;DR

This paper introduces a two-stage framework for text generation that separates high-level semantic planning from low-level word sequencing. By using a Generative Adversarial Network (GAN) to model the distribution of pre-trained sentence embeddings—acting as "neural outlines"—the authors successfully guide an autoregressive decoder to produce coherent, diverse text while avoiding the common "posterior collapse" problem found in Variational Autoencoders (VAEs).

The Challenge of Global Planning in Text Generation

Generating long, coherent sequences of text is a fundamental challenge in NLP. Standard autoregressive models, like traditional Language Models, generate text word-by-word. While they excel at local grammar, they often lose the "big picture" or high-level structure because they lack a global plan.

Researchers previously turned to Variational Autoencoders (VAEs) to introduce a global latent variable to guide generation. However, VAEs for text famously suffer from posterior collapse: the decoder becomes so powerful at predicting the next word based on previous words that it completely ignores the latent variable, rendering the "global plan" useless. This paper seeks a way to provide strong, informative high-level conditioning that the model cannot ignore.

Methodology: Decoupling Planning and Generation

The authors propose a decoupled approach consisting of three main components:

Model Architecture and Interpolation
  1. The Sentence Encoder: A pre-trained, general-purpose sentence encoder (specifically a GRU-based model trained on multiple tasks) maps sentences into a fixed-length 2048-dimensional vector space.
  2. The Outline Generator (GAN): Instead of a VAE, the framework uses a Wasserstein GAN with Gradient Penalty (WGAN-GP). The GAN is trained to model the distribution of these 2048-dim sentence vectors. Once trained, the GAN can sample new "neural outlines"—latent vectors that represent the semantic essence of a sentence.
  3. The Conditional Decoder: A GRU-based language model is trained to reconstruct the original sentence given its vector representation. To ensure the decoder is robust to the GAN's outputs (which might be slightly "off-manifold"), Gaussian noise is injected into the vectors during training.

Novel Interpolation: The authors also introduce a gradient-based optimization technique for interpolating between two sentences. Instead of moving in a straight line in the latent space (which often passes through "empty" regions), they use the decoder’s gradient to ensure the path stays in areas of high data density, resulting in more natural-looking semantic transitions.

Datasets

The model was evaluated across several diverse datasets:

  • SNLI (Stanford Natural Language Inference): Used for unconditional generation and a conditional task (generating a premise given a hypothesis and a logical relationship).
  • BookCorpus: A large collection of unpublished novels, used for modeling stylistic and narrative text.
  • WMT15: The English fraction of the News translation dataset.
  • SentiCap & Amazon Reviews: Used for conditional generation tasks based on sentiment.

Key Insights

  • Decoupling is Key: By separating the learning of the latent space (GAN) from the word-level generation (Decoder), the authors bypass the optimization hurdles that cause posterior collapse in VAEs.
  • Neural Outlines as Blueprints: The latent vector acts as a "neural outline." It doesn't just provide a hint; it effectively constrains the decoder to a specific semantic neighborhood.
  • Noise as a Bridge: Adding isotropic Gaussian noise to the embeddings during decoder training is crucial. It acts as a regularizer that forces the decoder to generalize to the approximate distributions produced by the GAN.
  • Overcoming "Holes" in Latent Space: High-dimensional latent spaces are often sparse. Their gradient-based "walking" technique shows that we can navigate the manifold more effectively by listening to what the decoder "thinks" is a valid sentence.

Results

  • Quantitative Quality: The model outperformed the Adversarially Regularized Autoencoder (ARAE) in terms of Forward and Reverse Perplexity (FPPL/RPPL). It remained competitive with state-of-the-art LMs even at high sampling temperatures where other models typically fall apart.
  • Human Evaluation: In A/B testing, humans preferred the "Neural Outline" samples for grammaticality and topicality, especially at higher temperatures.
  • Conditional Success: The model showed impressive results in the NLI task. For example, given the hypothesis "no one is dancing" and the label "contradiction," the model generated "a group of people playing guitar hero on a stage."
  • Smooth Interpolations: Qualitative results showed that the model could transition smoothly between two unrelated sentences in a way that felt semantically grounded.

Limitations and Future Work

  • Mode Collapse in Constraints: In some conditional tasks, the model occasionally fell into a form of "mode collapse," where it satisfied the condition by adding trivial, repetitive details.
  • Complexity of Benchmarks: While successful on SNLI and BookCorpus, the authors suggest moving toward more challenging benchmarks like MultiNLI to test the limits of the semantic representations.
  • Hybrid Objectives: Initial experiments combining Mean Squared Error (MSE) with adversarial objectives didn't yield immediate gains, but further exploration of the interaction between the GAN and the autoregressive component remains a promising research path.

Summary updated

Abstract

Recent progress in deep generative models has been fueled by two paradigms -- autoregressive and adversarial models. We propose a combination of both approaches with the goal of learning generative models of text. Our method first produces a high-level sentence outline and then generates words sequentially, conditioning on both the outline and the previous outputs. We generate outlines with an adversarial model trained to approximate the distribution of sentences in a latent space induced by general-purpose sentence encoders. This provides strong, informative conditioning for the autoregressive stage. Our quantitative evaluations suggests that conditioning information from generated outlines is able to guide the autoregressive model to produce realistic samples, comparable to maximum-likelihood trained language models, even at high temperatures with multinomial sampling. Qualitative results also demonstrate that this generative procedure yields natural-looking sentences and interpolations.
BibTeX
@inproceedings{subramanian2018towards,
  author = {Sandeep Subramanian and Sai Rajeswar and Alessandro Sordoni and Adam Trischler and Aaron C. Courville and Chris Pal},
  title = {{Towards Text Generation with Adversarially Learned Neural Outlines}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=BJWxHOZ_bS},
}
Figure