A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks

Nikunj Saunshi, Sadhika Malladi, Sanjeev Arora

Large language models (LLMs) like GPT-3 exhibit remarkable proficiency at downstream tasks—such as sentiment analysis or topic classification—even when they are not explicitly trained for them. While this zero-shot and few-shot success is well-documented empirically, the theoretical "why" has remained largely elusive.

There is a massive conceptual gap between a language model's pretraining objective (minimizing cross-entropy for next-word prediction) and the goal of downstream tasks (e.g., determining if a movie review is positive or negative). This paper seeks to bridge that gap by exploring the intuitive connection between predicting the next word and classifying text, and mathematically formalizing how success on the former guarantees success on the latter.

Theoretical Framework and Methodology

The authors approach the problem through a rigorous mathematical lens, focusing on three primary concepts:

  1. Natural Task Reformulation: The core hypothesis is that most classification tasks can be viewed as "sentence completion" problems. For instance, sentiment analysis is akin to asking a model to complete the prompt "This movie is..." with words like "great" or "terrible." The authors formalize these as natural tasks: tasks that can be solved linearly using the true conditional distribution of words.
  2. Mathematical Guarantees: The paper proves a crucial theorem: if a language model is $\epsilon$-close to the optimal cross-entropy (log-perplexity), the features it learns can solve these natural tasks with an error bound of $\mathcal{O}(\sqrt{\epsilon})$. This provides a rare, quantitative link between a model's pretraining perplexity and its downstream classification accuracy.
  3. Conditional Mean Features: The authors identify a specific type of feature—the expected word embedding under the model's predicted distribution—and demonstrate that these features are theoretically potent for linear classification.

Inspired by their analysis, the authors also propose a new training objective called Quad. This objective replaces the complex log-sum-exp (partition function) of the standard softmax with a quadratic approximation. The Quad objective has a closed-form solution that explicitly captures the "substitutability" of words (synonyms), providing a theoretical link between distributional semantics and modern deep learning.

Key Insights and Empirical Results

The authors validated their theory using a wide array of standard NLP benchmarks, including sentiment analysis (SST-2, IMDb) and topic classification (AG News, DBPedia), utilizing pretrained models like GPT-2.

  • Classification as Specialized Completion: The theoretical "aha moment" is that a linear classifier built on top of LM features is essentially just evaluating the probabilities of a few indicative words. The language model has already done the heavy lifting by learning the underlying text distribution.
  • Linear Probing Success: On datasets like SST-2 and AG News, a simple linear classifier applied to fixed GPT-2 features achieved 87.5% and 90.7% accuracy, respectively. This performance is remarkably close to full fine-tuning.
  • Minimalist Classification: In a striking demonstration of the theory, using the probabilities of just two words (like ":)" and ":(") as features yielded 78.7% accuracy on the SST dataset. This proves that LMs concentrate task-relevant information into specific, highly discriminative word probabilities.
  • Empirical Bound Verification: By training models to different levels of convergence, the authors empirically confirmed the $\mathcal{O}(\sqrt{\epsilon})$ trend predicted by their theorem. They also rigorously tested their mathematical assumptions regarding the model's internal mechanics, such as the behavior of the log partition function.
Scatter plots demonstrating the strong correlation between true and predicted log partition functions across SST, AG News, and WebText datasets.

Furthermore, the newly introduced Quad objective performed comparably to standard cross-entropy on several tasks, demonstrating that these theoretical insights can lead to viable, mathematically motivated algorithms.

Limitations and Future Work

While the paper provides a strong foundational theory, it notes a few limitations:

  • The Linearity Gap: The theory suggests a strict linear relationship between the model's internal features and the conditional mean features. However, in practice with models like GPT-2, this holds only weakly. The authors leave it to future work to explain why raw features often outperform theoretically derived ones.
  • Scope of Tasks: The current analysis is restricted to text classification. It does not yet cover more complex downstream tasks like text summarization, logical reasoning, or multi-hop question answering.
  • Black-Box Assumption: The paper treats the model architecture largely as a black box. A deeper dive into how specific architectures, particularly Transformers, provide the necessary inductive biases to reach these $\epsilon$-optimal solutions remains an open question for future research.

Summary updated

Abstract

Autoregressive language models, pretrained using large text corpora to do well on next word prediction, have been successful at solving many downstream tasks, even with zero-shot usage. However, there is little theoretical understanding of this success. This paper initiates a mathematical study of this phenomenon for the downstream task of text classification by considering the following questions: (1) What is the intuitive connection between the pretraining task of next word prediction and text classification? (2) How can we mathematically formalize this connection and quantify the benefit of language modeling? For (1), we hypothesize, and verify empirically, that classification tasks of interest can be reformulated as sentence completion tasks, thus making language modeling a meaningful pretraining task. With a mathematical formalization of this hypothesis, we make progress towards (2) and show that language models that are $\epsilon$-optimal in cross-entropy (log-perplexity) learn features that can linearly solve such classification tasks with $\mathcal{O}(\sqrt{\epsilon})$ error, thus demonstrating that doing well on language modeling can be beneficial for downstream tasks. We experimentally verify various assumptions and theoretical findings, and also use insights from the analysis to design a new objective function that performs well on some classification tasks.
BibTeX
@inproceedings{saunshi2021mathematical,
  author = {Nikunj Saunshi and Sadhika Malladi and Sanjeev Arora},
  title = {{A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=yRH3Tu3b3jE},
}

Loading related content...

Figure