State-of-the-art neural summarization models are notoriously data-hungry. As Hu et al. observe, deep learning architectures require massive volumes of paired data to effectively learn how to generate coherent, abstractive text. However, outside of standard news datasets with built-in headlines, high-quality, human-authored reference summaries are exceptionally scarce.
In specialized domains—such as community question-answering (Chakraborty et al.), noisy speech transcripts (Shang et al.), product reviews (Amplayo et al.), and project-specific source code (Xie et al.)—collecting gold-standard human labels is either prohibitively expensive or cognitively taxing. To bypass this annotation bottleneck, a prominent research direction focuses on training and evaluating summarization models using alternative supervision signals, synthetic data generation, and unsupervised geometric objectives.
Distant Supervision: Leveraging Environmental Proxies
Instead of waiting for humans to write summaries, we can look for existing environmental signals or metadata that correlate with content importance. This approach, known as distant supervision, treats external metrics as proxy labels for training.
For example, when summarizing online conversations, Kano et al. leverage social media popularity metrics (such as Reddit karma or upvotes) as a distant label. The intuition is straightforward: sentences or comments that receive high engagement are highly likely to contain the central, salient points of a discussion. By filtering out the inherent noise of social media metrics, models can learn to identify summary-worthy text without any explicit human annotation.
A similar intuition applies to document metadata. Isonuma et al. demonstrate that document classification categories can serve as "pseudo-rough" summaries. By training a model multi-taskally to both classify a document's topic and extract sentences, the model learns to prioritize sentences that carry the highest thematic weight. In community question-answering forums, Ishigaki et al. show that the structural relationship between user questions and corresponding answers provides a rich, unlabeled supervision signal that can guide extractive summarizers when gold labels are missing.
[Raw Text / Conversation] ───► [Extract High-Engagement Features] (e.g., Upvotes, Category Tags)
│
▼
[Distant Supervision Signal]
│
▼
[Train Summarization Model]
Synthetic Data: Engineering Training Pairs from Within
When the environment does not naturally provide proxy labels, we can algorithmically engineer synthetic training pairs directly from the source text. The goal is to create "silver-standard" datasets that mimic the structural complexity of human summaries.
One powerful method is to use parts of the document itself as a proxy target. Chen et al. introduce an adversarial domain adaptation framework that utilizes "artificial titles"—salient sentences extracted directly from target documents—to serve as proxy summaries. This allows a model trained on a data-rich domain (like news) to adapt its abstractive capabilities to a data-scarce domain (like social media) without requiring target-domain gold summaries.
For more complex tasks like query-focused summarization, creating synthetic data requires pairing documents with specific user intents. Xu et al. address this by using large language models to "reverse engineer" the implicit questions that generic summaries answer. By generating document-query-summary triplets from existing generic corpora, they create massive, high-quality training sets for query-focused tasks.
Similarly, the authors of OpineSum (Louis et al.) use textual entailment to automatically identify consensus opinions across review clusters, generating silver-standard summaries to self-train transformer models. To maximize data efficiency, Tan et al. leverage external knowledge bases like ConceptNet to guide a weakly-supervised model, matching state-of-the-art abstractive performance using less than 1% of the standard training data.
Unsupervised Summarization: Finding the Mathematical Consensus
The most radical departure from supervised learning is to eliminate training pairs entirely. Unsupervised summarization operates on the premise that a summary is a mathematical compression or "consensus" of the source text's latent space.
In an end-to-end neural setting, MeanSum (Chu et al.) demonstrates that an unsupervised abstractive model can be trained using an autoencoder framework. For a cluster of related documents (such as reviews of the same product), the model computes the latent representations of each document and extracts their mean vector. A decoder then reconstructs a fluent text summary from this average representation. Because the mean vector captures the shared semantic properties while averaging out unique details, the decoded output naturally functions as a consensus summary.
[Review 1 Vector] ──┐
[Review 2 Vector] ──┼─► [Compute Mean Vector] ─► [Decoder] ─► [Unsupervised Summary]
[Review 3 Vector] ──┘
Alternatively, graph-based approaches frame summarization as a centrality problem. Classic algorithms like TextRank build graphs where nodes are sentences and edges represent semantic similarity. To make these systems responsive to specific user constraints without training data, Kazemi et al. introduce Biased TextRank. By biasing the graph's random walks toward a specific query or viewpoint vector, the algorithm extracts highly relevant explanations and summaries completely unsupervised.
For multi-document scenarios, Shang et al. combine graph-based multi-sentence compression with submodular optimization to generate abstractive meeting summaries, proving that structural and linguistic constraints alone can yield highly readable summaries from noisy, ungrammatical inputs.