Overview
While Large Language Models (LLMs) are becoming increasingly fluent, they continue to suffer from a "misinformation problem"—unintentionally generating factually false statements, often referred to as hallucinations. In sensitive, high-stakes contexts like healthcare, these hallucinations can be dangerous.
CARE-MI is a new Chinese-language benchmark specifically designed to catch hallucinations and evaluate misinformation in LLMs within the domain of maternity and infant care. By providing 1,612 expert-verified questions and a framework for automated "judgment models," the researchers demonstrate that while state-of-the-art models like GPT-4 lead the pack, even the best AI still falls significantly short of human medical experts in providing truthful, long-form advice.
The Evaluation Gap
The authors identify three major gaps in current LLM evaluation methods:
- Format: Most benchmarks rely on simple multiple-choice or "cloze" (fill-in-the-blank) tasks. These formats do not reflect how people actually interact with LLMs in the real world, which is typically through open-ended questions.
- Language: Misinformation research and evaluation benchmarks are heavily skewed toward English, leaving a significant void for high-quality Chinese benchmarks.
- Domain Specificity: There is a lack of specialized datasets for the maternity and infant care domain. This is a highly "knowledge-intensive" area where parents frequently seek urgent, reliable advice.
Methods and Dataset Construction
To address these gaps, the authors developed a rigorous, multi-stage pipeline to create the CARE-MI benchmark.
- Data Sourcing: The team pulled raw data from two medical Knowledge Graphs (BIOS, CPubMed) and two Medical Multiple-Choice Question Answering (MCQA) datasets (MLEC-QA, MEDQA).
- Synthetic Generation: They utilized a "QA-to-Declarative" (QA2D) approach to transform question-answer pairs into true statements. False statements were then generated via negation or entity replacement.
- Question Generation: Using the LLM ChatYuan, the researchers generated both True/False (T/F) and Open-Ended (OE) questions based on the synthesized statements.
- Knowledge Retrieval: To provide "ground truth" context for each question, they used BM25 to retrieve relevant supporting paragraphs from Chinese Wikipedia and medical textbooks.
- Expert Filtering: Medical experts reviewed 5,779 synthetic samples, filtering them down to a final dataset of 1,612 high-quality, non-controversial questions.
To reduce the reliance on expensive human grading during model evaluation, the researchers also fine-tuned "judgment models" (such as LLaMA-13B-T) to mimic human scoring based on two key metrics: Correctness and Interpretability.
Key Insights and Results
The researchers evaluated several prominent models on the CARE-MI benchmark, including GPT-4, GPT-3.5-turbo, ChatGLM-6B, MOSS, and BELLE.
- GPT-4 Sets the Benchmark, but Humans Still Win: GPT-4 achieved the highest correctness score (0.867), significantly outperforming the best Chinese open-source models (LLaMA-13B-T at 0.709 and MOSS-16B-SFT at 0.671). However, despite its strong performance, GPT-4 still lags behind the Human Baseline (0.938).
- The "Open-Ended" Struggle: All models performed significantly worse on open-ended questions compared to True/False questions. While most models could achieve roughly 0.8 correctness on T/F tasks, only the GPT family managed to maintain a score above 0.6 for open-ended reasoning.
- The "Truthfulness" Gap in Fine-Tuning: Increasing the size of instruction-following data during fine-tuning improves a model's fluency and "helpfulness," but it does not necessarily improve its truthfulness. In some cases, more instruction-tuning led to more confident-sounding misinformation.
- Correctness vs. Interpretability: The study found a strong linear correlation ($R^2 = 0.834$) between how correct a model is and how well it explains its reasoning. However, high interpretability can be a double-edged sword: a model that provides a detailed, highly readable, but factually wrong explanation is much more likely to deceive a user.
- Automated Judgment is Viable: The fine-tuned LLaMA-13B judgment model proved to be a reliable proxy for human experts, achieving a high Pearson correlation (0.868) with human correctness scores.
Limitations and Future Work
The authors acknowledge a few limitations to their benchmark. First, medical knowledge is subject to temporal decay; what is considered "correct" today may become outdated as clinical techniques advance. Second, because the benchmark is expert-constructed, it may not perfectly capture the idiosyncratic ways real-world users (e.g., anxious parents) phrase their concerns. Finally, while currently limited to maternity and infant care, the authors suggest this pipeline could be adapted for other sensitive domains, such as legal or financial advice.
Summary updated