TL;DR
This paper introduces a novel task called "scientific attribution," which aims to link specific linguistic expressions within a scientific paper to the citations they refer to. By developing a machine learning approach to automatically identify these attributions, the authors demonstrate that this information significantly improves performance on a downstream discourse analysis task, highlighting its practical utility in understanding scientific literature.
What Problem Does This Paper Address?
Scientific papers are built upon the work of others, and understanding who is being referenced when is crucial for tasks like information retrieval, summarization, and even understanding the flow of arguments. While formal citations (e.g., "[Author, Year]") are explicit, scientific discourse often uses more subtle linguistic cues – like pronouns ("they") or noun phrases ("this approach") – to refer to cited work. Existing methods primarily focus on identifying formal citations or resolving pronoun coreference in a general sense. This paper tackles the gap of automatically identifying when these less explicit linguistic expressions refer to specific cited papers, the current paper, or neither. This is important because ignoring these implicit references can lead to an incomplete understanding of how ideas are being attributed and built upon within a scientific text.
Methods
The authors frame the problem as a scientific attribution task, which involves classifying "markables" (definite descriptions, "work" nouns, pronouns) to identify their referent. They propose a machine learning approach that trains two separate classifiers: one for attributing a markable to a specific citation and another for attributing it to the CURRENT-PAPER.
Key aspects of their methodology include:
- Fixed Referent Set: Unlike traditional anaphora resolution, the possible referents are known upfront: items in the reference list, the current paper, or a "no specific paper" category.
- Multi-label Classification: A markable can be attributed to multiple citations simultaneously.
- Feature Engineering: They extract a rich set of features from POS-tagged text, including:
- Properties of the markable and the potential citation (e.g., type of markable, citation style).
- Distance measures between the markable and potential referents (in words, sentences, paragraphs).
- Contextual features like citation counts within the paragraph/paper and section headings.
- A "Hobbs' prediction" feature to identify the closest potential referent based on sentence structure.
- Two-Stage Approach: They train separate models for attribution to citations and attribution to the current work, recognizing that the linguistic cues might differ.
Datasets
- Training and Evaluation Data: The authors created an annotated corpus from 10 articles within the CmpLg (Computation and Language archive), totaling 4290 data points (markables). This corpus was annotated based on written guidelines, and high inter-annotator agreement (Krippendorff's alpha > 0.8) was achieved.
- Extrinsic Evaluation Data: For evaluating the utility of their attribution task, they used the corpus from Teufel and Moens (2002), which consists of 80 computational linguistics conference papers (around 12,000 sentences) tagged for Argumentative Zoning (AZ).
Key Insights and Ideas
- Implicit Attribution is Crucial: The paper strongly argues that understanding implicit references (beyond formal citations) is vital for a deeper comprehension of scientific discourse.
- Context Matters: Features related to citation frequency within the document and the proximity of the markable to potential referents proved highly informative.
- Pronouns are Easiest: Resolving attribution for pronouns was significantly easier than for other markable types, suggesting they are more reliably used to refer to prior work.
- Utility in Downstream Tasks: The core insight is that automatically identifying scientific attributions isn't just an academic exercise; it directly benefits other NLP tasks, such as discourse analysis.
Key Results
- Intrinsic Evaluation: Their machine learning approach achieved a high agreement score (Krippendorff's alpha = 0.68) with human annotations for the scientific attribution task, rising to 0.71 when considering only pronouns. This significantly outperformed baseline methods.
- Extrinsic Evaluation (Argumentative Zoning): Incorporating the scientific attribution features into an Argumentative Zoning classifier led to a notable improvement in performance. The Kappa statistic increased from 0.45 to 0.48, and the Macro-F measure improved from 0.50 to 0.53. This improvement was particularly pronounced for categories like "OTHER" and "AIM," which are directly related to attribution.
Limitations and Future Work
- Data Skew: The annotated data exhibited a skewed distribution, with a large majority of markables attributed to the current paper (69%). This is a common challenge in classification tasks.
- Feature Importance: While they identified key features, the paper suggests further exploration into feature interactions and potentially more sophisticated feature extraction methods.
- Future Work: The authors explicitly plan to investigate the use of scientific attribution information for citation function classification (e.g., determining if a citation is supportive, critical, or background). They also believe similar improvements could be seen in other discourse annotation tasks within scientific literature.
Summary updated