Are GANs overkill for NLP?

David Alvarez-Melis, Vikas K Garg, Adam Tauman Kalai

Generative Adversarial Networks (GANs) have undeniably revolutionized Computer Vision, producing hyper-realistic images that are often indistinguishable from reality. Yet, in Natural Language Processing (NLP), GANs have historically struggled to achieve the same level of dominance. While researchers often attribute this to the "discreteness" of text—which makes backpropagating through specific word choices difficult—this paper investigates a more fundamental, objective-level question: Are we even optimizing for the right thing when applying GANs to text?

In NLP, the standard approach is Maximum Likelihood Estimation (MLE), which minimizes cross-entropy or perplexity to train models like GPT-4. GANs, conversely, aim to minimize "distinguishability," training a generator to produce outputs so realistic that a discriminator network cannot tell them apart from the training data. This research provides a theoretical framework proving that, for sequential discrete data, these two seemingly distinct goals are mathematically equivalent. Consequently, using GANs for NLP may be a computationally expensive and roundabout way to achieve what MLE already does more efficiently.

The Theoretical Bridge: Likelihood vs. Distinguishability

The authors bridge the gap between adversarial training and likelihood maximization through several key theoretical frameworks:

Log-Linear Equivalence: The paper demonstrates that for log-linear models—a family that includes n-grams and neural networks with a softmax output layer—the gradient of the KL-divergence (the MLE objective) is exactly the "advantage vector" of the features used by a discriminator. In essence, maximizing likelihood and minimizing distinguishability are two sides of the same coin. If a GAN successfully converges, it often lands at the exact same optimal point an MLE-trained model would have reached.

The Partition Function Problem: This equivalence highlights exactly why GANs are necessary for images but overkill for text. In Computer Vision, computing the exact likelihood of a high-dimensional, real-valued image is incredibly difficult because it requires calculating an often intractable "partition function." GANs elegantly bypass this requirement. However, because text is discrete and sequential, NLP models can compute conditional likelihoods (the probability of the next word) highly efficiently. The primary advantage of a GAN is therefore rendered irrelevant in the text domain.

Next-Token Distinguishability: To handle the complexity of long sequences, the authors introduce a novel model where the discriminator evaluates the next token predicted by the generator rather than the entire sentence. This aligns the adversarial objective more closely with autoregressive language modeling.

Boosting Likelihood via Discriminators

To formalize this relationship, the authors provide a polynomial-time algorithm that uses a "weak" discriminator to iteratively improve a model's likelihood. This "boosting reduction" proves that if a discriminator can find any flaw in a generative model, that exact information can be used to directly increase the model's likelihood.

Key Theoretical Result: The authors prove (Theorem 1) that a model can be made "$\epsilon$-indistinguishable" from the true distribution in time proportional to $1/\epsilon^2$.

Empirical Validation: The paper substantiates its theory using a pre-trained SeqGAN (with a vocabulary size of 5,000 and a maximum sequence length of 20). By "boosting" the generator using the discriminator's feedback, the authors directly improved the model's Negative Log-Likelihood (NLL). The performance gap between the original and boosted models was lower-bounded exactly as predicted by their theoretical lemmas.

Limitations and Future Directions

While the theoretical link is robust, the authors acknowledge several nuances:

  • Convergence and Stability: Making GANs converge is notoriously difficult. MLE is significantly more stable, reinforcing it as the superior choice for sequential discrete data.
  • Unidirectional Focus: The efficient reduction relies heavily on the unidirectional, autoregressive nature of current language models. It remains unclear if these results apply as cleanly to non-sequential generative tasks.
  • Perceptual Quality: The paper acknowledges that maximizing likelihood does not always perfectly correlate with human-perceived "quality," a persistent challenge in generative modeling.

Ultimately, this work elegantly explains the "tractability gap" between vision and language. By proving that a discriminator is essentially just a tool to find where a model's likelihood is low, the authors show that when likelihood can be computed directly—as it can in NLP—adversarial networks are simply overkill.

Summary updated

Abstract

This work offers a novel theoretical perspective on why, despite numerous attempts, adversarial approaches to generative modeling (e.g., GANs) have not been as successful for certain generation tasks, particularly sequential tasks such as Natural Language Generation, as they have in others, such as Computer Vision. In particular, on sequential data such as text, maximum-likelihood approaches are significantly more utilized than GANs. We show that, while it may seem that maximizing likelihood is inherently different than minimizing distinguishability, this distinction is largely an artifact of the limited representational capacity of the model family, for a wide class of adversarial objectives. We give a theoretical model in which minimizing KL-divergence (i.e., maximizing likelihood) is a more efficient approach to effectively minimizing the same distinguishability criteria that adversarial models seek to optimize. Reductions show that minimizing distinguishability can be seen as simply boosting likelihood for certain families of models including n-gram models and neural networks with a softmax output layer. To achieve a full polynomial-time reduction, a novel next-token distinguishability model is considered. Some preliminary empirical evidence is also provided to substantiate our theoretical analyses.
BibTeX
@inproceedings{alvarezmelis2022are,
  author = {David Alvarez-Melis and Vikas K Garg and Adam Tauman Kalai},
  title = {{Are GANs overkill for NLP?}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=F02H1zNl213},
}
Figure