FUDGE: Controlled Text Generation With Future Discriminators

Kevin Yang, Dan Klein

Abstract

Controlled text generation is essential for tailoring large language models (LLMs) to specific tasks, yet existing methods often suffer from high computational costs or degradation in text quality. We present FUDGE (Future Discriminators for Generation), a lightweight, modular framework that steers pre-trained models without requiring fine-tuning or gradient-based optimization. By employing a binary classifier to predict whether a partial sequence will eventually satisfy a desired attribute, FUDGE re-weights the model's output probabilities in real-time. Our results demonstrate that FUDGE significantly outperforms baselines in poetry generation, topic control, and formality transfer while maintaining high fluency and efficiency.

Introduction

While modern LLMs excel at generating fluent text, "steering" them toward specific attributes—such as a particular poetic meter, rhyme scheme, or level of formality—remains a significant challenge. Traditional approaches typically fall into three categories: fine-tuning, which is computationally expensive and risks "distribution shift"; rejection sampling, which is highly inefficient; and gradient-based methods like PPLM, which are slow and require access to internal model weights.

FUDGE addresses these limitations by providing a method that is modular, efficient, and flexible. It requires only the output probabilities of the base model, making it compatible with "API-only" proprietary models.

Methodology: Steering with Future Discriminators

The core innovation of FUDGE is the application of Bayes' Rule to look ahead during the generation process. Instead of evaluating the current token in isolation, FUDGE asks: "Given the words written so far, how likely is it that this sentence will eventually satisfy the target attribute?"

The FUDGE architecture: combining base model probabilities with a future discriminator to select the next token.

The Inference Process

FUDGE decomposes the conditional probability $P(x_i | x_{1:i-1}, a)$ into the product of the base model's probability $P(x_i | x_{1:i-1})$ and a "future discriminator" $P(a | x_{1:i})$. The discriminator is a lightweight binary classifier (typically an LSTM) trained on text prefixes to predict attribute satisfaction. At each step, FUDGE filters the top candidate words from the base model through this discriminator, multiplying and renormalizing the probabilities to steer the output toward the goal.

This approach allows for multi-constraint generation. Because FUDGE operates in log-probability space, multiple discriminators (e.g., for both topic and rhyme) can be combined simply by summing their log-probabilities.

Experimental Results

The authors evaluated FUDGE across three distinct domains: poetry, topic control, and machine translation.

Poetry and Topic Control

In poetry generation (couplet completion), FUDGE achieved a 44% success rate in generating valid iambic pentameter, nearly matching the performance of Shakespearean text (45%) and vastly outperforming the gradient-based PPLM baseline (0%).

Quantitative results for poetry generation, showing FUDGE's superior success rate and diversity compared to baselines.

In topic control tasks, FUDGE maintained higher fluency (4.30/5.0) than PPLM (4.05/5.0) while consistently generating on-topic content. Notably, FUDGE is highly efficient, generating 420 samples in just 15 minutes on a single GPU— a task that took PPLM several hours.

Formality in Machine Translation

For zero-shot formality transfer in Spanish-English translation, FUDGE successfully increased formality scores without the typical drop in translation quality (BLEU scores) seen in standard style-transfer baselines. This suggests that FUDGE preserves the underlying model's linguistic knowledge while effectively re-weighting its vocabulary.

Discussion and Limitations

FUDGE offers a powerful mechanism for zero-weight control, ensuring that the base model's fundamental understanding of grammar remains intact. However, the method is probabilistic and does not provide hard guarantees for constraints like exact syllable counts. Additionally, training the future discriminator requires a labeled dataset of the target attribute. Future work may explore combining FUDGE with reranking or rejection sampling to further tighten control for high-stakes applications.

Summary updated

Abstract

We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation. Given a pre-existing model G for generating text from a distribution of interest, FUDGE enables conditioning on a desired attribute a (for example, formality) while requiring access only to G's output logits. FUDGE learns an attribute predictor operating on a partial sequence, and uses this predictor's outputs to adjust G's original probabilities. We show that FUDGE models terms corresponding to a Bayesian decomposition of the conditional distribution of G given attribute a. Moreover, FUDGE can easily compose predictors for multiple desired attributes. We evaluate FUDGE on three tasks -- couplet completion in poetry, topic control in language generation, and formality change in machine translation -- and observe gains in all three tasks.
BibTeX
@inproceedings{yang2021fudge,
  author = {Kevin Yang and Dan Klein},
  title = {{FUDGE: Controlled Text Generation With Future Discriminators}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=xL3eGB5FGg},
}
Figure