Matching Citation Text and Cited Spans in Biomedical Literature: a Search-Oriented Approach

Arman Cohan, Luca Soldaini, Nazli Goharian

Abstract

This paper addresses the challenge of automatically linking citation sentences (citances) in biomedical literature to the specific text spans they reference in original articles. By framing the task as an information retrieval problem and employing domain-specific query expansion via the Unified Medical Language System (UMLS), the authors demonstrate a 25.9% improvement in matching accuracy over standard baselines.

Introduction

In the rapidly expanding field of biomedical research, keeping pace with new findings is a significant challenge. Automated summarization techniques often rely on "citances"—sentences in citing articles that describe a previous work. However, these citations can be inaccurate or lack context. This paper, presented at NAACL 2015, addresses the "citance-reference span matching" problem: automatically linking a citation back to the specific text spans it references in the original article.

The Problem: Epistemic Value Drift

Citations often suffer from epistemic value drift, where nuanced findings (e.g., "X might inhibit Y") are cited as definitive facts ("X inhibits Y"). Additionally, citations frequently strip away experimental conditions or specific data points. To facilitate fact-checking and high-quality summarization, systems must identify the "ground truth" sentences in the reference article.

The task of linking a citing sentence to its corresponding reference span in the original article.

Methodology: An Information Retrieval Approach

The authors frame this task as a search problem where the citance acts as a "query" and the reference article is the "document collection."

  1. Indexing Spans: To capture multi-sentence context, the researchers indexed every possible span of one to five consecutive sentences in the reference article.
  2. Retrieval Model: The Vector Space Model (VSM) was found to be the most consistent performer among various standard IR models.
  3. Query Reformulation: The core innovation involved enriching the citance query using the Unified Medical Language System (UMLS). Strategies included:
  4. UMLS-reduce: Filtering for medical terms.
  5. Noun Phrases (NP): Focusing on core biological concepts.
  6. UMLS-expand: Adding synonyms to bridge terminology gaps (e.g., matching "Lats2" with "tumor suppressor LATS2").

Datasets and the Challenge of Human Agreement

The system was evaluated using the TAC 2014 Biomedical Summarization track dataset. A significant finding was the inherent difficulty of the task; four domain experts showed remarkably low agreement on the "correct" reference spans. As shown in the table below, full agreement among all annotators was achieved in only 2 out of 313 cases.

Human annotator agreement levels, showing that full agreement is extremely rare.

Key Results

The combination of Noun Phrases (NP) and UMLS Expansion yielded the best performance, achieving a 25.9% relative improvement in F-1 score and a 47.8% improvement in precision over the baseline. While the absolute F-1 scores remained below 0.20, this is largely attributed to the subjective nature of the task and low human agreement.

Conclusion

By treating citation matching as an IR problem and leveraging domain-specific ontologies, this work provides a robust framework for linking scientific claims to their origins. Future research will likely explore deeper semantic matching to better understand the logical intent behind citations.

Summary updated

Abstract

Citation sentences (citances) to a reference article have been extensively studied for summarization tasks. However, citances might not accurately represent the content of the cited article, as they often fail to capture the context of the reported findings and can be affected by epistemic value drift. Following the intuition behind the TAC (Text Analysis Conference) 2014 Biomedical Summarization track, we propose a system that identifies text spans in the reference article that are related to a given citance. We refer to this problem as citance-reference spans matching. We approach the problem as a retrieval task; in this paper, we detail a comparison of different citance reformulation methods and their combinations. While our results show improvement over the baseline (up to 25.9%), their absolute magnitude implies that there is ample room for future improvement.
BibTeX
@inproceedings{cohan2015matching,
  author = {Arman Cohan and Luca Soldaini and Nazli Goharian},
  title = {{Matching Citation Text and Cited Spans in Biomedical Literature: a Search-Oriented Approach}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
  doi = {10.3115/v1/n15-1110},
}
Figure