Introduction
While Large Language Models (LLMs) demonstrate impressive capabilities, understanding what they truly "know" remains a challenge. Traditional benchmarks like LAMA (Language Model Analysis) often rely on a narrow set of prompts, leading to results skewed by "prompt luck." Furthermore, these benchmarks typically focus solely on accuracy, ignoring critical dimensions like consistency and reliability. To address these limitations, researchers have introduced BELIEF, a multifaceted framework designed to probe the parametric memory of language models more rigorously using MyriadLAMA, a massive dataset of 6.5 million diverse prompts.
The BELIEF Framework
The BELIEF framework moves beyond simple accuracy to evaluate knowledge across three core dimensions:
- Accuracy (Acc@1): Measures whether the model identifies the correct factual target.
- Consistency: Evaluates if the model provides the same answer when the same question is phrased in different ways.
- Reliability (Calibration): Assesses whether a model's internal confidence aligns with its actual likelihood of being correct.
To adapt this probing for modern decoder-only LLMs, the authors utilize BELIEF-ICL (In-Context Learning). This approach provides models with a few examples and a specific instruction to predict a masked token in one word, ensuring the evaluation is fair across different model architectures.
MyriadLAMA: Scaling Factual Probing
At the heart of this study is MyriadLAMA, a dataset that significantly expands the scope of factual probing. By leveraging GPT-4 to paraphrase relational templates and incorporating entity aliases from Wikidata, the researchers increased the number of prompts from roughly 24,000 in LAMA-UHN to over 6.4 million.
This scale allows for a much more robust analysis of how sensitive models are to linguistic variations and provides the statistical power needed to measure consistency and reliability accurately.
Key Insights
The study reveals several critical findings regarding how LLMs store and recall facts:
- The Instruction Tuning Tax: Surprisingly, instruction-tuning—while improving a model's ability to follow directions—often decreases its raw factual recall accuracy. This suggests that the tuning process might "overwrite" or make it harder to access some of the factual knowledge absorbed during pre-training.
- Memorized vs. Recallable Knowledge: There is a significant gap between what a model has stored in its weights and what it can successfully output. Using an "Oracle" metric (the best possible prompt for each fact), models performed much better than their average. This indicates that "forgetting" in LLMs is often a retrieval failure caused by suboptimal prompting.
- Data Quality Over Model Size: Smaller models trained on high-quality, "textbook-like" data, such as Phi-3-mini (3.8B), can outperform much larger models like Llama2-7B-IT in zero-shot knowledge retrieval.
Experimental Results
- Prompt Sensitivity: BERT-based models showed a 30% performance gap between their average and maximum accuracy across different templates, highlighting the fragility of current probing methods.
- Consistency and Reliability: Most models struggled with consistency, often failing to recognize a fact when the phrasing changed slightly. In terms of reliability, BERT models tended to be overconfident, while Llama 3 models were often underconfident in zero-shot settings, though their calibration improved with more in-context examples.
Conclusion and Future Work
The BELIEF benchmark and MyriadLAMA dataset provide a new standard for evaluating factual knowledge in LLMs. By highlighting the importance of prompt diversity and multifaceted evaluation, this work paves the way for more robust and reliable AI systems. Future research will focus on expanding the variety of factual relationships covered and developing more computationally efficient ways to utilize the massive MyriadLAMA dataset.
Summary updated