TL;DR
Large Language Models (LLMs) often struggle with questions containing false assumptions, leading to "hallucinated" answers. This paper introduces a novel approach that not only identifies these false assumptions but also provides interpretable explanations by breaking them down into atomic components, leading to more accurate and human-understandable responses.
What Problem Does This Paper Address?
People frequently ask questions that are built upon incorrect premises or beliefs. For instance, asking "Which countries border Kansas?" implies Kansas has an international border, which is false. LLMs, despite their impressive capabilities, often fail to recognize these false assumptions. Instead of pointing out the flawed premise, they tend to generate confident but incorrect or nonsensical answers, a phenomenon known as hallucination. This problem is exacerbated by the LLMs' overconfidence, where they "make up" answers rather than admitting ignorance. Existing work has focused on identifying unanswerable questions or verifying facts, but this paper tackles the specific challenge of identifying and answering questions with false assumptions in an interpretable way.
Methods
The authors propose a multi-stage approach to tackle this problem:
- Transforming Questions into Statements: Questions are first converted into declarative statements to simplify the subsequent analysis. This is achieved using few-shot prompting with LLMs like GPT-4.
- Retrieving Evidence: To combat hallucinations and ground the analysis, relevant documents are retrieved from external sources (like Wikipedia) using search engines and sentence ranking models.
- Generating and Validating Atomic Assumptions: This is the core novelty. Instead of just a binary "yes/no" for false assumptions, the approach breaks down the question into smaller, atomic assumptions. These atomic assumptions are then validated (true or false) using retrieved evidence. This process not only helps in identifying the overall false assumption but also provides a clear, human-readable explanation by pinpointing which specific atomic assumption is incorrect.
- Interpretable Answer Generation: The validated atomic assumptions (both true and false) are used to construct an interpretable answer that directly addresses the false premise of the original question.
The paper also explores several baselines, including fact verification, supervised fine-tuning of models like RoBERTa, and direct prompting of LLMs with and without retrieved evidence.
Datasets
The research utilizes three existing datasets for questions with false assumptions:
- (QA)² (Kim et al., 2023): Contains questions from Google and crowdworker-generated answers.
- CREPE (Yu et al., 2023b): Derived from the ELI5 subreddit, with crowdworker-identified false assumptions.
- FalseQA (Hu et al., 2023): Consists of synthetic questions with intentionally crafted false assumptions.
Additionally, they leverage BoolQ and FEVER for cross-domain transfer learning experiments.
Key Insights and Ideas
- False assumptions are distinct from false facts: Identifying a false assumption in a question is not the same as verifying a false factual claim. A question can be factually sound but still contain a false underlying assumption.
- Atomic assumptions are key to interpretability: Breaking down complex questions into smaller, verifiable atomic assumptions is crucial for both accurate identification and providing human-understandable explanations.
- Retrieval is essential, but generation needs careful handling: While retrieving external evidence significantly improves performance, LLMs generating their own evidence can be detrimental due to their tendency to hallucinate.
- Smaller models can benefit significantly: The atomic assumption generation and validation approach shows promise even for smaller LLMs, making it more accessible.
Key Results
The proposed approach, which generates and validates atomic assumptions, consistently outperforms baselines and previous state-of-the-art methods across all three datasets.
- State-of-the-art performance: The approach achieves state-of-the-art results on (QA)², CREPE, and FalseQA, with accuracy scores reaching up to 80.81% (GPT-4o on FalseQA for identifying with question and retrieved evidence) and 85.10% (GPT-4o on FalseQA for generating and validating atomic assumptions using the statement with retrieved evidence).
- Interpretability: The method provides succinct, human-readable interpretations by pinpointing specific false assumptions, a capability lacking in previous non-interpretable approaches.
- Efficiency: Despite requiring evidence retrieval and atomic assumption generation, the approach is more computationally cost-effective than using large, proprietary models for direct interpretation generation.
- Evidence benefits: Incorporating retrieved evidence consistently improves performance across all LLMs and datasets, with improvements being statistically significant.
Limitations and Future Work
- LLM Prompting Dependency: The methodology relies heavily on LLM prompting, which can limit reproducibility. The authors mitigate this by providing detailed implementation information and prompts.
- Retrieval System Dependency: The performance is influenced by the underlying retrieval system, which could be a bottleneck. However, this modularity allows for easy upgrades.
- Computational Overhead: While more cost-effective than some alternatives, the approach still incurs additional computational costs for retrieval and assumption processing compared to simple question-only identification.
- Future Work: The authors suggest exploring the validation of atomic assumptions for generating more robust and interpretable answers, and further investigating the generation of atomic assumptions with smaller LLMs. They also acknowledge the need for more comprehensive benchmarks and error analysis.
Summary updated