TL;DR
Large Language Models (LLMs) often hallucinate facts, especially when asked multiple questions at once (the "multi-problem setting"). This paper introduces MAC-Tuning, a novel fine-tuning method that separates answer prediction from confidence estimation. This allows LLMs to better identify and express uncertainty, significantly improving their reliability and accuracy in complex, multi-question scenarios.
What Problem Does This Paper Address?
LLMs are increasingly used in applications requiring factual accuracy, but they have a well-documented tendency to "hallucinate" – generating plausible-sounding but incorrect information. While much research has focused on mitigating this in single-question settings by understanding LLM's internal "knowledge boundaries," this paper tackles the more challenging "multi-problem setting." This setting involves an LLM answering several distinct questions simultaneously, often within a single prompt. This is harder because the LLM must manage multiple lines of reasoning, avoid context overshadowing between questions, and synthesize information cohesively. The increasing efficiency of this multi-problem approach makes improving its reliability crucial.
Methods
The core innovation is MAC-Tuning, a two-step fine-tuning process designed for the multi-problem setting:
- Data Construction: The authors first create "Multi-Problem Datasets" by combining individual questions from existing datasets. They then automatically label the LLM's output for each question as "I am sure" (if correct) or "I am unsure" (if incorrect). This creates two types of training data:
- Multiple QA Pair (DMultQA): Standard question-answer pairs for the multi-problem input.
- Multiple QA-Confidence Pair (DMultQA,C): Pairs where the LLM is prompted to express its confidence in its answers.
- Two-Step Fine-Tuning:
- Step 1 (Answer Prediction): The LLM is fine-tuned on DMultQA to learn how to predict answers for the multi-problem input.
- Step 2 (Confidence Estimation): The LLM is then fine-tuned on DMultQA,C to learn how to express its confidence in its answers, using the "I am sure" / "I am unsure" labels.
This separation is key: it allows the LLM to first learn to answer questions and then learn to assess its own certainty, enhancing its awareness of its knowledge boundaries.
Datasets
The authors evaluate their method on a variety of datasets, categorized by whether the questions are independent or sequential:
- Independent Setting:
- CoQA (Conversational Question Answering)
- ParaRel (Factual knowledge with relationships)
- GSM (Grade School Math problems)
- MMLU (Massive Multitask Language Understanding)
- Sequential Setting:
- MTI-Bench (Multi-task inference benchmark)
- SQA (Answering sequences of inter-related questions on HTML tables)
Key Insights and Ideas
- Separation is Key: The most significant insight is that separating the learning of answer prediction from confidence estimation is crucial for robust multi-problem reasoning. Trying to learn both simultaneously (as in the "Merge-AC" baseline) is less effective.
- Confidence as a Signal: The "I am sure" vs. "I am unsure" labels, derived from factual correctness, serve as a powerful signal for the LLM to understand its own knowledge gaps.
- Generalization: MAC-Tuning demonstrates good generalization capabilities, performing well even on out-of-domain datasets.
- Human Alignment: Human evaluation shows a substantial gap in factual accuracy between answers the model labels "I am sure" (89.2%) and those it labels "I am unsure" (41.2%), indicating that the learned confidence is a genuine reflection of answer quality.
Key Results
- Performance Boost: MAC-Tuning consistently outperforms baseline methods, achieving up to a 25% gain in Average Precision (AP) across various datasets and LLM backbones.
- Improved Calibration: The method significantly reduces Expected Calibrated Error (ECE), meaning the model's expressed confidence better matches its actual accuracy.
- Accuracy Gains: MAC-Tuning consistently improves accuracy by up to 45.8% compared to the base model, averaging around 23.7%.
- Scalability: The method shows robustness and scalability even with a large number of combined questions (n=10 and n=15), maintaining high accuracy.
Limitations and Future Work
- Prompt Sensitivity: The authors acknowledge that slight changes in prompts might affect results, a common limitation in prompt-based LLM studies.
- Experimental Scope: Due to cost and time constraints, they used a subset of experiments and datasets. Future work could expand this to more datasets and LLMs.
- Underlying Reasons: The paper suggests that future work should explore other potential reasons for experimental results in this new setting.
- Difficulty Mixing: They propose investigating whether mixing questions of varying difficulty and diversity in multi-problem settings leads to better scaling behavior and model generalization.
- Cross-Task Transfer: Further research could explore how fine-tuning on multi-problem settings affects performance on single-problem tasks and vice-versa.
Summary updated