Abstract
This paper addresses the challenge of automatically linking citation sentences (citances) in biomedical literature to the specific text spans they reference in original articles. By framing the task as an information retrieval problem and employing domain-specific query expansion via the Unified Medical Language System (UMLS), the authors demonstrate a 25.9% improvement in matching accuracy over standard baselines.
Introduction
In the rapidly expanding field of biomedical research, keeping pace with new findings is a significant challenge. Automated summarization techniques often rely on "citances"—sentences in citing articles that describe a previous work. However, these citations can be inaccurate or lack context. This paper, presented at NAACL 2015, addresses the "citance-reference span matching" problem: automatically linking a citation back to the specific text spans it references in the original article.
The Problem: Epistemic Value Drift
Citations often suffer from epistemic value drift, where nuanced findings (e.g., "X might inhibit Y") are cited as definitive facts ("X inhibits Y"). Additionally, citations frequently strip away experimental conditions or specific data points. To facilitate fact-checking and high-quality summarization, systems must identify the "ground truth" sentences in the reference article.
Methodology: An Information Retrieval Approach
The authors frame this task as a search problem where the citance acts as a "query" and the reference article is the "document collection."
- Indexing Spans: To capture multi-sentence context, the researchers indexed every possible span of one to five consecutive sentences in the reference article.
- Retrieval Model: The Vector Space Model (VSM) was found to be the most consistent performer among various standard IR models.
- Query Reformulation: The core innovation involved enriching the citance query using the Unified Medical Language System (UMLS). Strategies included:
- UMLS-reduce: Filtering for medical terms.
- Noun Phrases (NP): Focusing on core biological concepts.
- UMLS-expand: Adding synonyms to bridge terminology gaps (e.g., matching "Lats2" with "tumor suppressor LATS2").
Datasets and the Challenge of Human Agreement
The system was evaluated using the TAC 2014 Biomedical Summarization track dataset. A significant finding was the inherent difficulty of the task; four domain experts showed remarkably low agreement on the "correct" reference spans. As shown in the table below, full agreement among all annotators was achieved in only 2 out of 313 cases.
Key Results
The combination of Noun Phrases (NP) and UMLS Expansion yielded the best performance, achieving a 25.9% relative improvement in F-1 score and a 47.8% improvement in precision over the baseline. While the absolute F-1 scores remained below 0.20, this is largely attributed to the subjective nature of the task and low human agreement.
Conclusion
By treating citation matching as an IR problem and leveraging domain-specific ontologies, this work provides a robust framework for linking scientific claims to their origins. Future research will likely explore deeper semantic matching to better understand the logical intent behind citations.
Summary updated