CiteME: Can Language Models Accurately Cite Scientific Claims?

Ori Press, Andreas Hochlehnert, Ameya Prabhu, Vishaal Udandarao, Ofir Press, Matthias Bethge

Abstract

Thousands of new scientific papers are published each month. Such information overload complicates researcher efforts to stay current with the state-of-the-art as well as to verify and correctly attribute claims. We pose the following research question: Given a text excerpt referencing a paper, could an LM act as a research assistant to correctly identify the referenced paper? We advance efforts to answer this question by building a benchmark that evaluates the abilities of LMs in citation attribution. Our benchmark, CiteME, consists of text excerpts from recent machine learning papers, each referencing a single other paper. CiteME use reveals a large gap between frontier LMs and human performance, with LMs achieving only 4.2-18.5% accuracy and humans 69.7%. We close this gap by introducing CiteAgent, an autonomous system built on the GPT-4o LM that can also search and read papers, which achieves an accuracy of 35.3% on CiteME. Overall, CiteME serves as a challenging testbed for open-ended claim attribution, driving the research community towards a future where any claim made by an LM can be automatically verified and discarded if found to be incorrect.

Introduction

As the volume of scientific literature grows exponentially, researchers increasingly rely on Large Language Models (LLMs) to synthesize information and track the state-of-the-art. However, a significant barrier to the adoption of LLMs as reliable research assistants is their tendency to "hallucinate" citations—providing plausible-sounding but entirely fabricated references for scientific claims. To address this, we must determine if LLMs can accurately attribute claims to existing literature. This paper introduces CiteME, a rigorous benchmark designed to evaluate citation attribution, and CiteAgent, an autonomous system designed to bridge the performance gap between AI and human researchers.

The Limitations of Current Benchmarks

Existing citation datasets, such as FullTextPeerRead, ACL-200, and RefSeer, often fail to provide a rigorous test for modern LLMs. Analysis of these datasets reveals that only approximately 21% of their excerpts are "reasonable" for testing attribution. The remainder are either too ambiguous (where multiple papers could plausibly fit the claim) or trivial (where the paper title or authors are explicitly mentioned in the text).

To create a more effective testbed, we developed CiteME. This benchmark consists of 130 manually curated instances from recent machine learning papers (2008–2024). Crucially, we filtered out any instances that frontier models like GPT-4o could answer correctly from internal memory alone. This ensures that CiteME specifically tests a model's ability to perform active retrieval and reasoning rather than simple memorization.

CiteAgent: An Autonomous Research Assistant

Standard LLMs often fail at citation tasks because they rely solely on static training data. To overcome this, we developed CiteAgent, an agentic system that mimics the workflow of a human researcher. CiteAgent is equipped with three primary tools:

  1. search(query, sort): Interfaces with the Semantic Scholar database to find relevant literature.
  2. read(ID): Extracts and processes the full text of a PDF to verify claims against evidence.
  3. select(ID): Finalizes the attribution once sufficient evidence is gathered.

By allowing the model to actively search and read, CiteAgent moves beyond "guessing" and toward verifiable research assistance.

Experimental Results and Analysis

Our evaluation on the CiteME benchmark highlights a stark disparity between human and machine performance:

  • Human Performance: Human researchers achieved 69.7% accuracy, typically identifying the correct paper in under 40 seconds.
  • Standard LLMs: Without external tools, frontier models struggled significantly, with accuracy ranging from 4.2% to 18.5%. Retrieval-only models like SPECTER2 achieved 0% accuracy, demonstrating that vector similarity alone is insufficient for precise attribution.
  • CiteAgent Performance: By utilizing the search-and-read loop, CiteAgent (powered by GPT-4o) reached 35.3% accuracy. While this represents a significant improvement over standalone models, a substantial "reasoning gap" remains.
  • The Memorization Factor: Newer models like OpenAI’s o1-preview achieved 61.3% accuracy. However, evidence suggests this success is heavily bolstered by the model having already encountered these papers during its training phase, rather than through active reasoning on the provided excerpt.

Failure Modes and Costs

When CiteAgent fails, it is often due to "premature stopping" (32% of errors)—where the agent understands the excerpt but stops searching too early—or focusing on the wrong segment of the text (50% of errors). Additionally, high-accuracy attribution comes with a cost; a single CiteAgent check using GPT-4o costs approximately $1.20 and takes an average of 83 seconds.

Conclusion and Future Work

CiteME provides a challenging new standard for evaluating the reliability of AI-generated citations. While CiteAgent demonstrates the potential of agentic systems to narrow the human-AI gap, several hurdles remain. Future work must address the limited context windows of open-source models (like LLaMA-3) which hinder their ability to read full PDFs, as well as the technical fragility of current API-based agents. Ultimately, the goal is to develop systems capable of automatically verifying every claim an LLM makes, ensuring that scientific communication remains grounded in verifiable fact.

Summary updated

Abstract

Thousands of new scientific papers are published each month. Such information overload complicates researcher efforts to stay current with the state-of-the-art as well as to verify and correctly attribute claims. We pose the following research question: Given a text excerpt referencing a paper, could an LM act as a research assistant to correctly identify the referenced paper? We advance efforts to answer this question by building a benchmark that evaluates the abilities of LMs in citation attribution. Our benchmark, CiteME, consists of text excerpts from recent machine learning papers, each referencing a single other paper. CiteME use reveals a large gap between frontier LMs and human performance, with LMs achieving only 4.2-18.5% accuracy and humans 69.7%. We close this gap by introducing CiteAgent, an autonomous system built on the GPT-4o LM that can also search and read papers, which achieves an accuracy of 35.3\% on CiteME. Overall, CiteME serves as a challenging testbed for open-ended claim attribution, driving the research community towards a future where any claim made by an LM can be automatically verified and discarded if found to be incorrect.
BibTeX
@inproceedings{press2024citeme,
  author = {Ori Press and Andreas Hochlehnert and Ameya Prabhu and Vishaal Udandarao and Ofir Press and Matthias Bethge},
  title = {{CiteME: Can Language Models Accurately Cite Scientific Claims?}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=S9Qrrxpy6z},
}
Figure