Generating Text from Language Models

Afra Amini, Ryan Cotterell, John Hewitt, Clara Meister, Tiago Pimentel

Introduction

While Large Language Models (LLMs) have achieved remarkable success in natural language processing, the process of generating high-quality text remains a complex challenge. Practitioners often rely on a "dark science" of intuition and trial-and-error when selecting decoding algorithms. This paper provides a comprehensive tutorial framework that unifies the fragmented landscape of text generation, offering a formal understanding of why models fail and how specific techniques—such as sampling adapters and controlled generation—can mitigate these issues.

The Challenges of Out-of-the-Box Generation

Standard language models, when used without specific decoding strategies, frequently encounter three core failure modes:

  1. The "Tail" Problem: Models often assign non-zero probability to "garbage" or incoherent tokens. While the probability of a single bad token may be low, the cumulative probability of encountering at least one incoherent token increases significantly over long sequences.
  2. Repetition Loops: When attempting to find the most likely sequence (the mode), models often become trapped in repetitive, high-probability loops that lack semantic value.
  3. Lack of Control: Base models lack inherent mechanisms to ensure outputs adhere to specific constraints, such as maintaining a particular sentiment, avoiding toxicity, or following a specific syntactic structure.

A Unified Framework for Decoding

The paper frames language generation as the interaction between a scoring function (the objective) and a decoding algorithm. This distinction allows for a more structured approach to text production through several key methods:

Sampling Adapters

The authors propose a unified view of stochastic decoding where methods like nucleus (top-p) and top-k sampling are treated as functions $g$. These functions map the model's original distribution to a "locally adapted" distribution. By pruning the "tail" of low-probability tokens, these adapters effectively prevent the model from drifting into incoherence.

Controlled Generation via Bayes' Rule

To steer a model toward a specific attribute $c$ (e.g., "politeness"), the framework utilizes a Bayesian approach. The model's output is reweighted according to the formula: $p(y|c) \propto p(y)p(c|y)$. This allows practitioners to guide a massive, frozen language model using a relatively simple and efficient classifier $p(c|y)$.

Energy-Based and MCMC Methods

For complex constraints where autoregressive steering is insufficient—such as specific syntactic requirements—the paper suggests Markov-Chain Monte Carlo (MCMC) methods like Langevin Dynamics. These methods allow for sampling the entire sequence at once, rather than token-by-token, to satisfy global constraints.

Theoretical Insights

The Mathematics of Failure

A critical insight provided is the math of cumulative error. If a model has only a 0.1% chance of selecting an incoherent token at any given step, there is approximately a 20% chance that a 200-token sequence will contain at least one error. This mathematical reality explains why unfiltered sampling inevitably leads to gibberish in long-form generation.

Entropy and Task Type

The authors suggest that the "open-endedness" of a generation task is best described by the entropy of the true distribution. High-entropy tasks, such as creative storytelling, require stochastic sampling to maintain variety. Conversely, low-entropy tasks, such as machine translation, benefit more from deterministic search algorithms.

Efficiency of Classifiers

A key takeaway for controlled generation is that it is significantly more computationally and data-efficient to build a classifier to recognize a desired attribute than it is to fine-tune a massive model to generate that attribute natively.

The paper synthesizes several key findings from recent literature:

  • Nucleus Sampling as the Standard: Nucleus sampling has become the de facto baseline because it dynamically adjusts the probability cutoff based on the model's confidence, offering more flexibility than the fixed window of top-k sampling.
  • The Failure of the Mode: For many modern LMs, the "most likely" sequence (often found via beam search) is frequently repetitive and of lower qualitative value than sequences with slightly lower total probability.
  • MCMC for Hard Constraints: While Bayesian steering works well for attributes like sentiment, MCMC-based methods are significantly more effective for enforcing rigid syntactic or structural constraints.

Limitations and Future Outlook

Despite these advancements, several challenges remain. In controlled generation, errors in the classifier's estimate $p(c|y)$ can propagate and compound throughout a sequence. Furthermore, there is a significant "evaluation gap," as standard metrics like BLEU and ROUGE fail to capture nuances like faithfulness or interestingness. Finally, the authors emphasize the urgent need to expand generation research beyond English-centric models to account for the world's typologically diverse languages.

Summary updated

Abstract

An increasingly large percentage of natural language processing (NLP) tasks center around the generation of text from probabilistic language models. Despite this trend, techniques for improving or specifying preferences in these generated texts rely mostly on intuition-based heuristics. Further, there lacks a unified presentation of their motivations, practical implementation, successes and pitfalls. Practitioners must, therefore, choose somewhat blindly between generation algorithms—like top-p sampling or beam search—which can lead to wildly different results. At the same time, language generation research continues to criticize and improve the standard toolboxes, further adding entropy to the state of the field. In this tutorial, we will provide a centralized and cohesive discussion of critical considerations when choosing how to generate from a language model. We will cover a wide range of empirically-observed problems (like degradation, hallucination, repetition) and their corresponding proposed algorithmic solutions from recent research (like top-p sampling and its successors). We will then discuss a subset of these algorithms under a unified light; most stochastic generation strategies can be framed as locally adapting the probabilities of a model to avoid failure cases. Finally, we will then cover methods in controlled generation, that go beyond just ensuring coherence to ensure text exhibits specific desired properties. We aim for NLP practitioners and researchers to leave our tutorial with a unified framework which they can use to evaluate and contribute to the latest research in language generation.
BibTeX
@inproceedings{amini2023generating,
  author = {Afra Amini and Ryan Cotterell and John Hewitt and Clara Meister and Tiago Pimentel},
  title = {{Generating Text from Language Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-tutorials.4},
}
Figure