CARE-MI: Chinese Benchmark for Misinformation Evaluation in Maternity and Infant Care

Tong Xiang, Liangzhi Li, Wangyue Li, Mingbai Bai, Lu Wei, Bowen Wang, Noa Garcia

Overview

While Large Language Models (LLMs) are becoming increasingly fluent, they continue to suffer from a "misinformation problem"—unintentionally generating factually false statements, often referred to as hallucinations. In sensitive, high-stakes contexts like healthcare, these hallucinations can be dangerous.

CARE-MI is a new Chinese-language benchmark specifically designed to catch hallucinations and evaluate misinformation in LLMs within the domain of maternity and infant care. By providing 1,612 expert-verified questions and a framework for automated "judgment models," the researchers demonstrate that while state-of-the-art models like GPT-4 lead the pack, even the best AI still falls significantly short of human medical experts in providing truthful, long-form advice.

The Evaluation Gap

The authors identify three major gaps in current LLM evaluation methods:

  1. Format: Most benchmarks rely on simple multiple-choice or "cloze" (fill-in-the-blank) tasks. These formats do not reflect how people actually interact with LLMs in the real world, which is typically through open-ended questions.
  2. Language: Misinformation research and evaluation benchmarks are heavily skewed toward English, leaving a significant void for high-quality Chinese benchmarks.
  3. Domain Specificity: There is a lack of specialized datasets for the maternity and infant care domain. This is a highly "knowledge-intensive" area where parents frequently seek urgent, reliable advice.

Methods and Dataset Construction

To address these gaps, the authors developed a rigorous, multi-stage pipeline to create the CARE-MI benchmark.

The multi-stage pipeline used to create the CARE-MI benchmark.
  1. Data Sourcing: The team pulled raw data from two medical Knowledge Graphs (BIOS, CPubMed) and two Medical Multiple-Choice Question Answering (MCQA) datasets (MLEC-QA, MEDQA).
  2. Synthetic Generation: They utilized a "QA-to-Declarative" (QA2D) approach to transform question-answer pairs into true statements. False statements were then generated via negation or entity replacement.
  3. Question Generation: Using the LLM ChatYuan, the researchers generated both True/False (T/F) and Open-Ended (OE) questions based on the synthesized statements.
  4. Knowledge Retrieval: To provide "ground truth" context for each question, they used BM25 to retrieve relevant supporting paragraphs from Chinese Wikipedia and medical textbooks.
  5. Expert Filtering: Medical experts reviewed 5,779 synthetic samples, filtering them down to a final dataset of 1,612 high-quality, non-controversial questions.
Examples of true statements, True/False questions, and Open-Ended questions generated for the benchmark.

To reduce the reliance on expensive human grading during model evaluation, the researchers also fine-tuned "judgment models" (such as LLaMA-13B-T) to mimic human scoring based on two key metrics: Correctness and Interpretability.

Key Insights and Results

The researchers evaluated several prominent models on the CARE-MI benchmark, including GPT-4, GPT-3.5-turbo, ChatGLM-6B, MOSS, and BELLE.

  • GPT-4 Sets the Benchmark, but Humans Still Win: GPT-4 achieved the highest correctness score (0.867), significantly outperforming the best Chinese open-source models (LLaMA-13B-T at 0.709 and MOSS-16B-SFT at 0.671). However, despite its strong performance, GPT-4 still lags behind the Human Baseline (0.938).
  • The "Open-Ended" Struggle: All models performed significantly worse on open-ended questions compared to True/False questions. While most models could achieve roughly 0.8 correctness on T/F tasks, only the GPT family managed to maintain a score above 0.6 for open-ended reasoning.
  • The "Truthfulness" Gap in Fine-Tuning: Increasing the size of instruction-following data during fine-tuning improves a model's fluency and "helpfulness," but it does not necessarily improve its truthfulness. In some cases, more instruction-tuning led to more confident-sounding misinformation.
  • Correctness vs. Interpretability: The study found a strong linear correlation ($R^2 = 0.834$) between how correct a model is and how well it explains its reasoning. However, high interpretability can be a double-edged sword: a model that provides a detailed, highly readable, but factually wrong explanation is much more likely to deceive a user.
  • Automated Judgment is Viable: The fine-tuned LLaMA-13B judgment model proved to be a reliable proxy for human experts, achieving a high Pearson correlation (0.868) with human correctness scores.

Limitations and Future Work

The authors acknowledge a few limitations to their benchmark. First, medical knowledge is subject to temporal decay; what is considered "correct" today may become outdated as clinical techniques advance. Second, because the benchmark is expert-constructed, it may not perfectly capture the idiosyncratic ways real-world users (e.g., anxious parents) phrase their concerns. Finally, while currently limited to maternity and infant care, the authors suggest this pipeline could be adapted for other sensitive domains, such as legal or financial advice.

Summary updated

Abstract

The recent advances in natural language processing (NLP), have led to a new trend of applying large language models (LLMs) to real-world scenarios. While the latest LLMs are astonishingly fluent when interacting with humans, they suffer from the misinformation problem by unintentionally generating factually false statements. This can lead to harmful consequences, especially when produced within sensitive contexts, such as healthcare. Yet few previous works have focused on evaluating misinformation in the long-form (LF) generation of LLMs, especially for knowledge-intensive topics. Moreover, although LLMs have been shown to perform well in different languages, misinformation evaluation has been mostly conducted in English. To this end, we present a benchmark, CARE-MI, for evaluating LLM misinformation in: 1) a sensitive topic, specifically the maternity and infant care domain; and 2) a language other than English, namely Chinese. Most importantly, we provide an innovative paradigm for building LF generation evaluation benchmarks that can be transferred to other knowledge-intensive domains and low-resourced languages. Our proposed benchmark fills the gap between the extensive usage of LLMs and the lack of datasets for assessing the misinformation generated by these models. It contains 1,612 expert-checked questions, accompanied with human-selected references. Using our benchmark, we conduct extensive experiments and found that current Chinese LLMs are far from perfect in the topic of maternity and infant care. In an effort to minimize the reliance on human resources for performance evaluation, we offer off-the-shelf judgment models for automatically assessing the LF output of LLMs given benchmark questions. Moreover, we compare potential solutions for LF generation evaluation and provide insights for building better automated metrics.
BibTeX
@inproceedings{xiang2023care,
  author = {Tong Xiang and Liangzhi Li and Wangyue Li and Mingbai Bai and Lu Wei and Bowen Wang and Noa Garcia},
  title = {{CARE-MI: Chinese Benchmark for Misinformation Evaluation in Maternity and Infant Care}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
}
Figure