TL;DR
This paper introduces a system to aid researchers in writing survey papers by leveraging "reference information" – the relationships between cited and citing papers, and the context surrounding citations. The system extracts "reference areas" (fragments describing a paper's essence and its relation to others) and classifies citation types to help identify relevant papers and understand their similarities and differences, ultimately supporting the creation of comprehensive surveys.
What Problem Does This Paper Address?
The exponential growth of scientific literature makes it increasingly difficult for researchers to keep up with all relevant papers in a specific domain. Survey papers are crucial for navigating this information overload, but their creation is a time-consuming and labor-intensive process. The authors identify two key challenges in writing surveys: effectively collecting relevant papers and understanding their core contributions and how they relate to each other (similarities and differences). Existing methods for summarizing individual papers are insufficient for this multi-paper context, necessitating a new approach that focuses on inter-paper relationships.
Methods
The core of the proposed system lies in its use of "reference information." This includes:
- Reference Relationships: The explicit links between papers (e.g., citations).
- Reference Areas: Fragments of text within a citing paper where the author discusses the essence of a referred paper and its relationship to their own work. These are identified using cue words and phrases (e.g., "However," "In addition," "We propose").
- Reference Types: Classification of the reason for a citation. The paper defines three types:
- Type B: Referencing to build upon existing theories or methods.
- Type C: Referencing to compare with related works or highlight their problems. This type is emphasized as particularly valuable for summarization.
- Type O: Other types of references.
The system first extracts these reference areas and then classifies them into reference types. Type C references are particularly useful because their associated text often describes the referred paper's essence (similarities) and the citing paper's contribution (differences). The system then uses this information to build a "reference graph" that visually represents the relationships between papers within a domain.
Datasets
The authors used a database of 450 papers in TeX source format from the e-Print archive in computational linguistics for their experiments. They also manually identified 150 reference areas for evaluating reference area extraction and 382 reference areas for evaluating reference type identification.
Key Insights and Ideas
- Reference Information is Key: The paper argues that the context surrounding citations, not just the citation itself, holds crucial information for understanding paper relationships and summarizing domains.
- Type C Citations are Gold: Identifying "Type C" citations (those used for comparison or critique) is highlighted as a powerful way to uncover similarities and differences between papers, which are fundamental to writing a survey.
- Domain Specificity: Tracing only Type C reference relationships proved effective in collecting papers within the same domain, with 94% of such relationships linking papers in the same field.
- Visualizing Relationships: The system's ability to display a "reference graph" and associated abstracts/reference areas provides a valuable visual aid for researchers to grasp the landscape of a research domain.
Key Results
- Reference Area Extraction: The proposed method achieved an F-measure of 0.833, outperforming two baseline methods (0.534 and 0.575) that considered the whole paragraph or just the sentence with the citation, respectively.
- Reference Type Identification: The system achieved an accuracy of 90.1% for rule creation data and 83.0% for evaluation data in identifying reference types.
- Domain Collection: Tracing Type C references successfully collected papers within the same domain in 94% of cases.
Limitations and Future Work
The authors acknowledge that their current system is a "support system for writing a survey" and that the ultimate goal is "automatic survey generation." While the system effectively aids in collecting papers and understanding their relationships, the full automation of survey writing remains a future challenge. The paper suggests that the methods developed for reference area extraction and type identification are foundational steps towards this goal.
Summary updated