In-context Learning and Gradient Descent Revisited

Gilad Deutch, Nadav Magar, Tomer Bar Natan, Guy DarNAACL 2024 · DOI

TL;DR

This paper revisits the hypothesis that In-Context Learning (ICL) in large language models is equivalent to gradient descent (GD) optimization. The authors find surprisingly little evidence for this "strong" correspondence, showing that even untrained models exhibit similar levels of similarity to GD. They propose a "layer-causal" variant of GD that better aligns with ICL, suggesting a weaker, more nuanced relationship.

What Problem Does This Paper Address?

Large language models (LLMs) exhibit remarkable "in-context learning" (ICL) abilities, where they improve performance on a task by simply being shown a few examples within the input prompt. While this phenomenon is powerful, its underlying mechanism remains a mystery. A recent line of research has proposed that ICL is essentially a form of implicit gradient descent (GD) optimization, where the LLM learns by treating the provided examples as training data. However, much of this research has focused on simplified models and tasks. This paper aims to rigorously test this "strong ICL-GD correspondence" hypothesis on realistic NLP tasks and models, and to investigate potential discrepancies and alternative explanations.

Methods

The authors critically re-evaluate the evaluation metrics and baselines used in prior work that claimed a strong ICL-GD correspondence. They introduce a "layer-causal" gradient descent (LCGD) method. Unlike standard GD, where updates to a layer depend on gradients from the entire model, LCGD updates are based only on information from preceding layers, mimicking the information flow in ICL. This is achieved by projecting intermediate layer outputs to the vocabulary space and computing a loss at each layer, then updating layer parameters independently. They also propose a modified similarity metric, SimAM△, to better capture the similarity of update directions.

Datasets

The paper uses the following datasets for evaluation, consistent with prior work:

  • SST2 (Sentiment Analysis)
  • SST5 (Sentiment Analysis)
  • MR (Sentiment Analysis)
  • Subj (Sentiment Analysis)
  • AGNews (Topic Classification)
  • CB (Natural Language Inference)

Key Insights and Ideas

  • Untrained Models Mimic GD: Surprisingly, untrained LLMs achieve similarity scores to GD that are comparable to, or even better than, trained models. This strongly suggests that the observed similarities are not necessarily indicative of ICL being a form of learned optimization.
  • Layer Causality is Key: A core discrepancy between ICL and standard GD is "layer causality." In ICL, information flows sequentially, with later layers building on earlier ones. Standard GD, however, uses gradients from all layers to update any given layer, breaking this sequential causality.
  • Layer-Causal GD Shows Promise: A proposed "layer-causal" gradient descent (LCGD) method, which respects this sequential information flow, shows improved similarity to ICL compared to vanilla GD. This suggests that a more nuanced, layer-aware form of GD might be a better analogy for ICL.
  • Evaluation Metrics Matter: The authors highlight how certain evaluation metrics (specifically, the normalized SimAOU) can be misleading and overestimate similarity, especially when dealing with random noise or unnormalized vectors.

Key Results

  • The paper demonstrates that untrained transformers perform as well as, and sometimes better than, pre-trained models on similarity metrics between ICL and GD.
  • LCGD consistently improves similarity scores compared to vanilla GD across most tasks and metrics, particularly in attention map similarity (SimAM△).
  • While LCGD shows improvements, the absolute similarity scores remain relatively low, suggesting that even this modified approach doesn't fully capture the ICL mechanism.

Limitations and Future Work

  • Similarity Metrics: The current metrics only consider a specific correspondence where each layer performs GD. It's possible that the true mechanism involves a different kind of correspondence or that not all layers are involved in GD.
  • Datasets: The dataset collection is heavily skewed towards sentiment classification, and the CB dataset is very small, potentially leading to instability. Testing on a wider variety of tasks and models is needed.
  • LCGD Instantiation: The proposed LCGD is just one specific instantiation of layer-causal gradient descent. Better variants might exist.
  • Hyperparameter Tuning: The authors acknowledge that hyperparameter search, particularly for learning rates, could significantly impact the similarity scores and warrants further investigation.
  • Future Work: The authors suggest exploring better layer-causal variants, investigating the impact of different learning rates, and connecting the layer-causal approach to existing work on kernel regression and functional GD.

Summary updated

BibTeX
@inproceedings{deutch2024context,
  author = {Gilad Deutch and Nadav Magar and Tomer Bar Natan and Guy Dar},
  title = {{In-context Learning and Gradient Descent Revisited}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.58},
}
Figure