Sure, here's a blog-style summary of the paper "ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models":
TL;DR
This paper introduces CONTRADOC, the first large-scale, human-annotated dataset for detecting self-contradictions within documents. They evaluate several state-of-the-art Large Language Models (LLMs) on this dataset, finding that while GPT-4 shows promising performance, LLMs still struggle with nuanced self-contradictions, highlighting the need for further research in this area.
What Problem Does This Paper Address?
Detecting contradictions in text is a fundamental aspect of Natural Language Understanding (NLU). While much research has focused on identifying contradictions between separate texts (Natural Language Inference or NLI), understanding contradictions within a single document, known as self-contradictions, especially in long documents, remains an under-explored area. Humans also find it challenging to spot these inconsistencies, particularly when they are spread out across a lengthy text. This paper aims to bridge this gap by creating a dedicated dataset and evaluating LLMs' capabilities in this challenging task.
Methods
The authors developed a human-machine collaborative framework to create the CONTRADOC dataset. First, LLMs were used to automatically generate self-contradictions within existing documents. Then, human annotators verified these contradictions, ensuring their naturalness and labeling them with attributes like type (e.g., negation, numeric, factual) and scope (intra, local, global). To evaluate LLMs, they designed three tasks:
- Binary Judgment: A simple "Yes/No" question asking if a document contains a self-contradiction.
- Self-Contradiction Top-k: Asking the model to identify the top 5 most probable sentences indicating a self-contradiction.
- Judge then Find: A more complex task where the model first answers "Yes/No" and then provides the supporting evidence sentences if the answer is "Yes."
Datasets
The primary contribution is the CONTRADOC dataset, which consists of:
- CONTRADOC-POS: 449 self-contradictory documents.
- CONTRADOC-NEG: 442 non-contradictory documents.
These documents are sourced from three domains:
- News Articles: CNN-DailyMail
- Stories: NarrativeQA
- Wikipedia: WikiText
The dataset includes annotations for contradiction type, scope, and evidence, making it a rich resource for studying self-contradictions.
Key Insights and Ideas
- LLMs struggle with self-contradictions: Despite advancements, current LLMs, even powerful ones like GPT-4, are not fully reliable in detecting self-contradictions, especially those requiring nuanced understanding or context spread across a document.
- Objective contradictions are easier: LLMs perform better at detecting more objective self-contradictions (e.g., numeric discrepancies, negations) compared to subjective ones (e.g., emotional inconsistencies, differing perspectives). This is hypothesized to be due to LLMs being pre-trained on more fact-checking tasks.
- Document length isn't the primary hurdle: The study found that document length didn't significantly impact LLMs' ability to detect self-contradictions, suggesting that the nature of the contradiction and its type are more critical factors.
- Evidence matters: While models might correctly identify a document as contradictory, their ability to pinpoint the exact contradictory sentences (evidence) is often weaker, highlighting a gap between judgment and justification.
Key Results
- GPT-4 leads: GPT-4 demonstrated the best performance across all evaluation tasks, outperforming other LLMs like GPT-3.5, PaLM2, and LLaMAv2.
- Binary Judgment: Models achieved around 50% accuracy, indicating near-random performance, with low recall values suggesting they tend to predict "No" even when contradictions exist.
- Self-Contradiction Top-k: GPT-4 achieved a 70.2% Evidence Hit Rate (EHR), significantly higher than others. It also tended to find evidence earlier in the document.
- Judge then Find: GPT-4 achieved the highest "real accuracy" (R-acc(pos)) of 35.6%, and its evidence hit rate was a remarkable 92.7% when it predicted "Yes."
- Human Performance: Humans performed better than GPT-3.5 but not GPT-4 on the Judge then Find task, particularly in identifying evidence.
Limitations and Future Work
- Dataset Limitations: While CONTRADOC is comprehensive, some self-contradiction types have limited data points (e.g., only 25 for "Relation"). The dataset currently only includes English documents.
- LLM Limitations: Even GPT-4, while leading, is not perfect. It still struggles with certain types of contradictions and can be overly conservative in its predictions. The prompt format also seemed to affect GPT-3.5's performance.
- Future Work: The authors plan to investigate LLMs' capabilities in fixing self-contradictions in documents, which is a natural next step after detection. They also suggest further exploration into the interplay between contradiction types, document domains, and LLM performance.
Summary updated