TL;DR
This paper investigates how well Large Language Models (LLMs) memorize factual knowledge, finding they excel at popular facts but struggle with less common ones. They introduce "Adaptive Retrieval," a method that intelligently uses external knowledge sources only when the LLM is likely to be wrong, improving performance and efficiency.
What Problem Does This Paper Address?
LLMs have shown remarkable capabilities, but they often falter on tasks requiring deep factual knowledge. This is because encoding the vastness of world knowledge into their parameters (parametric knowledge) is challenging, especially for less frequently encountered information. The paper aims to understand when LLMs can be trusted to recall facts from their internal knowledge versus when they need external help (non-parametric knowledge, like retrieved text). They also explore how model scaling and retrieval augmentation impact this memorization.
Methods
The researchers conducted large-scale knowledge probing experiments using various LLMs, from smaller GPT-Neo and OPT models to larger GPT-3 variants. They evaluated these models on open-domain question answering (QA) tasks in a zero- or few-shot setting.
Key methodological contributions include:
- POPQA Dataset: A new dataset of 14,000 questions focusing on "long-tail" entities (less common facts) to better probe memorization limitations.
- Adaptive Retrieval: A novel approach that dynamically decides whether to use retrieval augmentation based on the popularity of the entity in the question. If the entity is popular, the LLM relies on its parametric knowledge; if it's less popular, retrieval is triggered. This is determined using a popularity threshold that is set independently for each relationship type.
- Analysis of Factors: Investigating the impact of subject entity popularity and relationship type on memorization.
Datasets
- POPQA: A newly created dataset of 14,000 open-domain QA questions focusing on long-tail entities. It was constructed by sampling knowledge triples from Wikidata and converting them into natural language questions, with popularity measured by Wikipedia page views.
- EntityQuestions: A publicly available open-domain QA dataset also known for its long-tail entity distribution, used for validation.
- Natural Questions (NQ): Used for comparison and context, particularly regarding entity popularity distributions.
Key Insights and Ideas
- Popularity is Key: LLMs are much better at memorizing facts about popular entities. The more frequently an entity is discussed (e.g., on the web), the more likely an LLM is to have learned about it.
- Scaling Has Limits for the Long Tail: While scaling up LLMs generally improves performance, it doesn't significantly boost the memorization of less popular, long-tail facts.
- Retrieval Augmentation is Powerful, But Not Always: Non-parametric memories (retrieved text) significantly improve performance, especially for smaller models and less popular facts. However, for popular facts, retrieval can sometimes hurt performance because the LLM has already memorized the answer and the retrieved context might be misleading or incorrect.
- Adaptive Retrieval is the Sweet Spot: The most effective approach is to intelligently combine parametric and non-parametric knowledge. Adaptive Retrieval achieves this by only retrieving external information when the LLM is likely to fail, leading to better accuracy and efficiency.
Key Results
- POPQA Performance: GPT-3 davinci-003 achieved 35% accuracy on POPQA without retrieval, while GPT-Neo 20B achieved 25%.
- Long-Tail Struggle: For the 4,000 least popular questions in POPQA, GPT-j 6B had 16% accuracy and GPT-3 davinci-003 had 19%, showing scaling's limited impact.
- Retrieval Boost: Retrieval-augmented LMs, particularly with Contriever, significantly outperformed unassisted LMs. A smaller GPT-Neo 2.7B with Contriever outperformed vanilla GPT-3 on the least popular questions.
- Adaptive Retrieval Gains: Adaptive Retrieval achieved up to 46.5% accuracy on POPQA with GPT-3 davinci-003, a 5.3% improvement over non-adaptive methods. It also reduced inference costs by up to 15% on EntityQuestions and up to 9% on POPQA.
- Misleading Retrieval: In about 10% of cases, retrieval augmentation actually caused GPT-3 davinci-003 to answer incorrectly, often due to low-quality or irrelevant retrieved text.
Limitations and Future Work
- Dataset Specificity: The study primarily used synthetic datasets (POPQA, EntityQuestions). The extent to which these findings apply to naturally occurring factual knowledge needs further investigation.
- Popularity Definition: The use of Wikipedia page views as a popularity proxy is a simplification and may not perfectly reflect real-world discussion frequency. Future work could explore alternative definitions.
- Other Knowledge Types: The findings are focused on factual knowledge. It's an open question whether similar principles apply to other knowledge types like commonsense reasoning.
- Controlled Experiments: The authors emphasize the need for controlled experiments to fully understand the interplay of scaling, retrieval, popularity, and relationship types.
- Bias and Stereotypes: The paper notes that LMs might rely on stereotypes for minority groups, and future work could investigate how retrieval augmentation can mitigate such biases.
Summary updated