ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models

Jierui Li, Vipul Raheja, Dhruv Kumar

Sure, here's a blog-style summary of the paper "ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models":

TL;DR

This paper introduces CONTRADOC, the first large-scale, human-annotated dataset for detecting self-contradictions within documents. They evaluate several state-of-the-art Large Language Models (LLMs) on this dataset, finding that while GPT-4 shows promising performance, LLMs still struggle with nuanced self-contradictions, highlighting the need for further research in this area.

What Problem Does This Paper Address?

Detecting contradictions in text is a fundamental aspect of Natural Language Understanding (NLU). While much research has focused on identifying contradictions between separate texts (Natural Language Inference or NLI), understanding contradictions within a single document, known as self-contradictions, especially in long documents, remains an under-explored area. Humans also find it challenging to spot these inconsistencies, particularly when they are spread out across a lengthy text. This paper aims to bridge this gap by creating a dedicated dataset and evaluating LLMs' capabilities in this challenging task.

Methods

The authors developed a human-machine collaborative framework to create the CONTRADOC dataset. First, LLMs were used to automatically generate self-contradictions within existing documents. Then, human annotators verified these contradictions, ensuring their naturalness and labeling them with attributes like type (e.g., negation, numeric, factual) and scope (intra, local, global). To evaluate LLMs, they designed three tasks:

  1. Binary Judgment: A simple "Yes/No" question asking if a document contains a self-contradiction.
  2. Self-Contradiction Top-k: Asking the model to identify the top 5 most probable sentences indicating a self-contradiction.
  3. Judge then Find: A more complex task where the model first answers "Yes/No" and then provides the supporting evidence sentences if the answer is "Yes."

Datasets

The primary contribution is the CONTRADOC dataset, which consists of:

  • CONTRADOC-POS: 449 self-contradictory documents.
  • CONTRADOC-NEG: 442 non-contradictory documents.

These documents are sourced from three domains:

  • News Articles: CNN-DailyMail
  • Stories: NarrativeQA
  • Wikipedia: WikiText

The dataset includes annotations for contradiction type, scope, and evidence, making it a rich resource for studying self-contradictions.

Key Insights and Ideas

  • LLMs struggle with self-contradictions: Despite advancements, current LLMs, even powerful ones like GPT-4, are not fully reliable in detecting self-contradictions, especially those requiring nuanced understanding or context spread across a document.
  • Objective contradictions are easier: LLMs perform better at detecting more objective self-contradictions (e.g., numeric discrepancies, negations) compared to subjective ones (e.g., emotional inconsistencies, differing perspectives). This is hypothesized to be due to LLMs being pre-trained on more fact-checking tasks.
  • Document length isn't the primary hurdle: The study found that document length didn't significantly impact LLMs' ability to detect self-contradictions, suggesting that the nature of the contradiction and its type are more critical factors.
  • Evidence matters: While models might correctly identify a document as contradictory, their ability to pinpoint the exact contradictory sentences (evidence) is often weaker, highlighting a gap between judgment and justification.

Key Results

  • GPT-4 leads: GPT-4 demonstrated the best performance across all evaluation tasks, outperforming other LLMs like GPT-3.5, PaLM2, and LLaMAv2.
  • Binary Judgment: Models achieved around 50% accuracy, indicating near-random performance, with low recall values suggesting they tend to predict "No" even when contradictions exist.
  • Self-Contradiction Top-k: GPT-4 achieved a 70.2% Evidence Hit Rate (EHR), significantly higher than others. It also tended to find evidence earlier in the document.
  • Judge then Find: GPT-4 achieved the highest "real accuracy" (R-acc(pos)) of 35.6%, and its evidence hit rate was a remarkable 92.7% when it predicted "Yes."
  • Human Performance: Humans performed better than GPT-3.5 but not GPT-4 on the Judge then Find task, particularly in identifying evidence.

Limitations and Future Work

  • Dataset Limitations: While CONTRADOC is comprehensive, some self-contradiction types have limited data points (e.g., only 25 for "Relation"). The dataset currently only includes English documents.
  • LLM Limitations: Even GPT-4, while leading, is not perfect. It still struggles with certain types of contradictions and can be overly conservative in its predictions. The prompt format also seemed to affect GPT-3.5's performance.
  • Future Work: The authors plan to investigate LLMs' capabilities in fixing self-contradictions in documents, which is a natural next step after detection. They also suggest further exploration into the interplay between contradiction types, document domains, and LLM performance.

Summary updated

Abstract

In recent times, large language models (LLMs) have shown impressive performance on various document-level tasks such as document classification, summarization, and question-answering. However, research on understanding their capabilities on the task of self-contradictions in long documents has been very limited. In this work, we introduce ContraDoc, the first human-annotated dataset to study self-contradictions in long documents across multiple domains, varying document lengths, self-contradictions types, and scope. We then analyze the current capabilities of four state-of-the-art open-source and commercially available LLMs: GPT3.5, GPT4, PaLM2, and LLaMAv2 on this dataset. While GPT4 performs the best and can outperform humans on this task, we find that it is still unreliable and struggles with self-contradictions that require more nuance and context. We release the dataset and all the code associated with the experiments (https://github.com/ddhruvkr/CONTRADOC).
BibTeX
@inproceedings{li2024contradoc,
  author = {Jierui Li and Vipul Raheja and Dhruv Kumar},
  title = {{ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.362},
}
Figure