Measuring and Narrowing the Compositionality Gap in Language Models

Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, Mike Lewis

Abstract

We investigate the ability of language models to perform compositional reasoning tasks where the overall solution depends on correctly composing the answers to sub-problems. We measure how often models can correctly answer all sub-problems but not generate the overall solution, a ratio we call the compositionality gap. We evaluate this ratio by asking multi-hop questions with answers that require composing multiple facts unlikely to have been observed together during pretraining. In the GPT-3 family of models, as model size increases we show that the single-hop question answering performance improves faster than the multi-hop performance does, therefore the compositionality gap does not decrease. This surprising result suggests that while more powerful models memorize and recall more factual knowledge, they show no corresponding improvement in their ability to perform this kind of compositional reasoning. We then demonstrate how elicitive prompting (such as chain of thought) narrows the compositionality gap by reasoning explicitly instead of implicitly. We present a new method, self-ask, that further improves on chain of thought. In our method, the model explicitly asks itself (and then answers) follow-up questions before answering the initial question. We finally show that self-ask's structured prompting lets us easily plug in a search engine to answer the follow-up questions, which additionally improves accuracy.

The Challenge of Multi-Hop Reasoning

While Large Language Models (LLMs) are exceptional at retrieving facts, their ability to combine those facts—compositional reasoning—remains a major hurdle. The authors define the compositionality gap as the failure rate where a model knows the individual facts (sub-questions) but fails to answer the complex question requiring their combination (e.g., "Who was the U.S. President when the person who directed Inception was born?").

Scaling is Not a Panacea

A primary finding of the research is that increasing model size does not naturally close this gap. As GPT-3 models scale from 0.35B to 175B parameters, they become significantly better at remembering individual facts. However, their ability to compose those facts stays stagnant, with the compositionality gap remaining at roughly 40% across the InstructGPT-3 family. This suggests that scaling increases the "database" size of the model but does not inherently improve its "reasoning processor."

The compositionality gap remains persistent across model sizes in both GPT-3 and InstructGPT-3 families.

Narrowing the Gap with Self-Ask Prompting

To address this, the researchers propose self-ask, a structured prompting technique. Unlike the free-form prose of "Chain of Thought" (CoT), self-ask uses a rigid scaffolding where the model must explicitly output follow-up questions and intermediate answers before providing the final response. This structure prevents the model from wandering into hallucinations and ensures a clean decomposition of the problem.

Comparison of direct prompting, Chain of Thought, and the proposed Self-Ask method.

Search Engine Integration

The structured nature of self-ask allows for easy integration with external tools. By intercepting the "Follow up" questions, the system can query a search engine API and feed the real-world result back to the model as an "Intermediate answer." This approach (SA+SE) significantly boosts accuracy, particularly on "search-proof" datasets like Bamboogle, where it reached 60% accuracy compared to a 17.6% baseline for direct prompting.

Key Results and Efficiency

  • Performance: Self-ask consistently outperforms Chain of Thought across multi-hop benchmarks like 2WikiMultiHopQA and Musique.
  • Confidence: The study found a strong correlation between a model's confidence in a fact (low perplexity) and its ability to use that fact in reasoning.
  • Speed: Self-ask is more efficient than other decomposition methods because it processes the entire reasoning chain in a single structured forward pass.

Conclusion

The persistence of the compositionality gap highlights that scaling alone is insufficient for complex reasoning. However, elicitive prompting methods like self-ask provide a clear path forward, especially when combined with external knowledge retrieval, to bridge the gap between factual recall and true compositional intelligence.

Summary updated

Abstract

We investigate the ability of language models to perform compositional reasoning tasks where the overall solution depends on correctly composing the answers to sub-problems. We measure how often models can correctly answer all sub-problems but not generate the overall solution, a ratio we call the compositionality gap. We evaluate this ratio by asking multi-hop questions with answers that require composing multiple facts unlikely to have been observed together during pretraining. In the GPT-3 family of models, as model size increases we show that the single-hop question answering performance improves faster than the multi-hop performance does, therefore the compositionality gap does not decrease. This surprising result suggests that while more powerful models memorize and recall more factual knowledge, they show no corresponding improvement in their ability to perform this kind of compositional reasoning. We then demonstrate how elicitive prompting (such as chain of thought) narrows the compositionality gap by reasoning explicitly instead of implicitly. We present a new method, self-ask, that further improves on chain of thought. In our method, the model explicitly asks itself (and then answers) follow-up questions before answering the initial question. We finally show that self-ask's structured prompting lets us easily plug in a search engine to answer the follow-up questions, which additionally improves accuracy.
BibTeX
@inproceedings{press2023measuring,
  author = {Ofir Press and Muru Zhang and Sewon Min and Ludwig Schmidt and Noah A. Smith and Mike Lewis},
  title = {{Measuring and Narrowing the Compositionality Gap in Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=feiAVaSXdb},
}
Figure