Introduction
In the rapidly evolving landscape of artificial intelligence and computer science, researchers face a significant challenge in keeping pace with the sheer volume of new literature. While human-authored survey articles provide invaluable syntheses, they are labor-intensive to produce and quickly become obsolete. Existing automated summarization systems, such as C-Lexrank, effectively identify salient information but often fail to produce coherent narratives. These systems typically generate "word salads"—sequences of high-value sentences that lack logical transitions and contain "dangling" references (e.g., pronouns without clear antecedents).
Surveyor is a novel automated system designed to address these limitations by generating readable, coherent survey articles. It achieves this through a dual-model approach: a Content Model for high-level structure and a Discourse Model for local linguistic clarity.
Methodology
Surveyor's architecture is built on two primary components that work in tandem to ensure both global and local coherence.
The Content Model: Global Structure
To determine the logical flow of a survey, Surveyor utilizes a Hidden Markov Model (HMM). The system first clusters sentences from input papers into distinct subtopics using the Louvain network-based clustering algorithm. By analyzing how human authors transition between these subtopics in existing literature, the HMM learns a transition matrix. This allows the system to mimic professional writing patterns, such as ensuring "Background" information precedes "Specific Methodology."
The Discourse Model: Local Coherence
To solve the problem of extracted sentences losing their meaning out of context, the authors introduced Minimum Independent Discourse Contexts (MIDC). Rather than extracting a single sentence in isolation, the system identifies the smallest set of preceding sentences required to make the target sentence fully understandable. This identification is governed by specific dependency rules based on coreference chains, discourse markers, and shared entities.
The final generation algorithm iteratively selects subtopics based on the HMM, identifies the most salient sentence within that subtopic using Lexrank, and incorporates its associated MIDC into the summary until the desired length is reached.
Experimental Setup
The system was evaluated using the ACL Anthology Network (AAN), focusing on 15 established computational linguistics topics, including Machine Translation and Sentiment Analysis. For each topic, the introduction sections of the 20 most-cited papers served as input. Surveyor was benchmarked against C-Lexrank (the scientific summarization standard) and G-FLOW (a state-of-the-art news summarizer). Performance was measured against 30 human-written "gold standard" summaries using ROUGE scores and human qualitative assessments.
Results and Discussion
Surveyor demonstrated substantial improvements over baseline systems across both automated and human metrics.
As shown in the performance data, Surveyor achieved a ROUGE-2 score of 0.19, significantly outperforming C-Lexrank's 0.05. This indicates a much higher fidelity in capturing the factual pairings and phrasing found in human-written surveys. Furthermore, human assessors preferred Surveyor’s output 67% of the time in head-to-head comparisons with C-Lexrank, noting a 36% increase in coherence on a 5-point scale.
Ablation studies (Surveyor HMM Only and Surveyor MIDC Only) reveal that both components are necessary for peak performance. While the HMM provides the "map" for the article, the MIDC ensures that individual "steps" remain intelligible.
Conclusion
Surveyor represents a significant advancement in automated scientific summarization by shifting the focus from mere information extraction to coherent synthesis. By learning from human authorial patterns and preserving local discourse context, it produces summaries that are both informative and readable. Future research will explore automating input paper selection and incorporating natural language generation techniques, such as sentence compression and rewriting, to further refine the output.
Summary updated