Introduction
Generating "Related Work" sections is a cornerstone of academic writing, yet it remains a significant challenge for automated systems. Traditional approaches often treat this as a standard summarization task at the sentence level. However, academic sentences are complex; they frequently blend transitions, author reflections, and specific fragments of cited research. Treating the entire sentence as a single unit often leads to "hallucinations" or the inclusion of irrelevant context. To address this, the CORWA dataset shifts the focus from full sentences to citation spans—the precise text fragments derived from cited papers.
Methodology: A Multi-Layered Annotation Scheme
The researchers introduced the Citation-Oriented Related Work Annotation (CORWA) framework, which employs a three-pronged approach to capture the nuances of literature reviews:
- Discourse Tagging: Sentences are categorized into roles such as
single_summ(summarizing one paper),narrative_cite(high-level observations), ortransition(connecting topics). - Citation Span Detection: The model identifies the exact tokens directly attributable to a specific cited work.
- Citation Type Recognition: Citations are classified as Dominant (detailed explanations) or Reference (brief, illustrative mentions).
To automate this process, the authors developed a Joint Related Work Tagger based on SciBERT. This model utilizes multi-task learning to simultaneously predict discourse labels, identify spans, and classify citation types.
Dataset and Analysis
The CORWA dataset comprises 927 manually annotated related work sections from NLP papers. To scale the training data, the authors created "Distant CORWA" by applying their tagger to over 11,000 additional sections from the S2ORC corpus.
Analysis of the dataset reveals a stark contrast between citation types. Dominant spans tend to be significantly longer, averaging 34.5 tokens, as they provide deep dives into specific methodologies or results. In contrast, Reference spans average only 8.2 tokens, serving as concise pointers to support broader claims.
Key Results and Evaluation
The SciBERT-based joint tagger demonstrated high efficacy, achieving a micro-F1 score exceeding 0.9 across all three tasks (discourse, type, and span detection). Beyond tagging performance, the researchers found that focusing on spans significantly improves downstream tasks:
- Retrieval Accuracy: Using specific spans as search queries rather than full sentences improved ROUGE recall when retrieving source sentences from cited papers, particularly for short "reference" citations.
- Human Preference: In blind evaluations, NLP graduate students preferred span-based generation over traditional sentence-based methods, citing higher relevance and quality.
Conclusion and Future Directions
The CORWA dataset demonstrates that citation spans are a superior unit of measurement for literature review generation compared to full sentences. While currently focused on the NLP domain, the proposed "bottom-up" iterative framework—generating spans, grouping them into discourse blocks, and weaving them with transitions—offers a path toward more human-like, controllable AI writing assistants. Future work will explore the generalizability of this approach across other scientific disciplines.
Summary updated