A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks

Nikunj Saunshi, Sadhika Malladi, Sanjeev Arora

Introduction

Large-scale language models (LMs), such as GPT and BERT, have demonstrated remarkable proficiency in downstream tasks like sentiment analysis and topic classification, often requiring little to no task-specific fine-tuning. Despite this empirical success, the underlying mathematical reason why a model trained on the simple objective of next-word prediction (minimizing cross-entropy) should naturally develop features useful for complex categorization remains a "black box." This paper provides a theoretical bridge between pretraining objectives and downstream performance, proving that improvements in log-perplexity mathematically guarantee improvements in classification accuracy.

Formalizing "Natural Tasks"

The authors bridge the gap between pretraining and downstream application by formalizing the concept of "Natural Tasks." They hypothesize that the majority of classification tasks can be reformulated as sentence completion problems. For instance, a sentiment analysis task for the sentence "I loved this movie" can be framed as predicting the most likely word to follow the prompt: "I loved this movie. This movie is ___."

By viewing classification as a subset of prediction—specifically, predicting a small, task-relevant subset of the vocabulary (e.g., "good" vs. "bad") within a specific context—the researchers align the downstream goal with the pretraining objective.

Theoretical Framework: The $O(\sqrt{\epsilon})$ Bound

The core contribution of this work is a mathematical proof establishing an upper bound on downstream error. The authors demonstrate that if a language model is $\epsilon$-optimal in terms of cross-entropy loss during pretraining, the features it learns can solve "natural" classification tasks with an error rate of at most $O(\sqrt{\epsilon})$ when using a simple linear classifier.

This relationship quantifies the intuition that "better language models make better classifiers." As the cross-entropy loss decreases during the pretraining phase, the model is effectively forced to organize its internal representations in a way that squeezes the error rate of downstream linear classifiers. This bound is influenced by a constant $B$, representing the "indicative word mass," which explains why prompting (e.g., adding "This movie is...") is so effective: it aligns the task more closely with the natural structure of the training data, thereby reducing the error bound.

Methodology and the Quad Objective

To further explore these internal representations, the authors introduced two key concepts:

  1. Conditional Mean Features: They proposed a method to extract features by taking the expected value of word embeddings for the predicted next-word distribution ($\Phi p_{f(s)}$). This suggests that the model's hidden states are intrinsically linked to the semantic structure of the vocabulary.
  2. The Quad Objective: To test the robustness of their theory, the researchers designed a new objective function called "Quad." Unlike the standard softmax-based cross-entropy, Quad uses a quadratic approximation. This allows for a closed-form solution for optimal word embeddings based on a "substitutability matrix." This matrix captures the idea that synonyms are substitutable because they share similar conditional probabilities across various contexts—a structure that linear classifiers are particularly adept at exploiting.

Empirical Results

The researchers validated their theory using standard NLP benchmarks, including SST-2, SST-5, AG News, IMDb, and DBPedia. They utilized the 117M GPT-2 model and custom models trained on IMDb and Amazon review corpora.

Linear Success

The study demonstrated that linear classifiers trained on top of fixed GPT-2 features perform remarkably well. For example, on the AG News dataset, fixed features achieved approximately 90.7% accuracy, compared to 94.5% achieved through full fine-tuning. This suggests that the bulk of the task-relevant information is already captured during pretraining.

Theory Verification

By plotting the logistic loss of downstream tasks against the cross-entropy of LM checkpoints at various stages of training, the authors empirically confirmed the square-root relationship ($O(\sqrt{\epsilon})$) predicted by their mathematical theorem. Furthermore, the experimental "Quad" objective performed comparably to standard cross-entropy on sentiment tasks, proving that their mathematical abstraction of LM mechanics is robust enough to guide the creation of functional models.

Discussion and Limitations

While this work provides a significant step toward demystifying language models, several questions remain:

  • Linearity vs. Deep Fine-tuning: The current analysis focuses on linear classifiers on fixed features. While this explains the success of "frozen" models, it does not fully account for the performance gains observed during deep fine-tuning.
  • Architectural Inductive Biases: The proof treats the model largely as a black box and does not explain why specific architectures, like Transformers, might be more efficient at capturing these features than others, such as LSTMs.
  • Task Complexity: The current guarantees are restricted to text classification. Extending this mathematical framework to generative tasks like summarization or translation remains a challenge for future research.

Conclusion

This research provides a rigorous mathematical foundation for the observed success of pretrained language models. By proving that minimizing cross-entropy naturally optimizes the features required for downstream classification, the authors move the field closer to a transparent understanding of how and why large-scale pretraining works.

Summary updated

Abstract

Autoregressive language models, pretrained using large text corpora to do well on next word prediction, have been successful at solving many downstream tasks, even with zero-shot usage. However, there is little theoretical understanding of this success. This paper initiates a mathematical study of this phenomenon for the downstream task of text classification by considering the following questions: (1) What is the intuitive connection between the pretraining task of next word prediction and text classification? (2) How can we mathematically formalize this connection and quantify the benefit of language modeling? For (1), we hypothesize, and verify empirically, that classification tasks of interest can be reformulated as sentence completion tasks, thus making language modeling a meaningful pretraining task. With a mathematical formalization of this hypothesis, we make progress towards (2) and show that language models that are $\epsilon$-optimal in cross-entropy (log-perplexity) learn features that can linearly solve such classification tasks with $\mathcal{O}(\sqrt{\epsilon})$ error, thus demonstrating that doing well on language modeling can be beneficial for downstream tasks. We experimentally verify various assumptions and theoretical findings, and also use insights from the analysis to design a new objective function that performs well on some classification tasks.
BibTeX
@inproceedings{saunshi2021mathematical,
  author = {Nikunj Saunshi and Sadhika Malladi and Sanjeev Arora},
  title = {{A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=yRH3Tu3b3jE},
}
Figure