Introduction
While Large Language Models (LLMs) have demonstrated impressive capabilities in English summarization, a significant "low-resource trap" persists for many of the world's languages. High-quality summarization data is expensive to produce via human annotation and often noisy when scraped from the web. This paper explores a novel solution: repurposing a century of professionally curated content from digitized newspaper archives. By extracting "front-page teasers"—short summaries written by editors to entice readers—researchers can generate high-quality, abstractive summarization datasets for languages like Hebrew, Icelandic, and Estonian.
Methodology: The Teaser-to-Article Pipeline
The core methodology involves a two-step automated pipeline designed to transform newspaper archives into gold-standard datasets.
- Teaser Identification: The system uses rule-based heuristics to scan front pages for teasers. These are identified by their characteristic brevity and the presence of explicit page references (e.g., "See page 4").
- Article Matching: Once a teaser is identified, the system must locate the corresponding article(s) on the referenced page. The authors evaluated three matching strategies: a simple TF-IDF word-overlap method, fine-tuned Sentence Transformers for semantic similarity, and zero-shot LLMs (such as Llama-3.3 70B).
A significant advantage of this approach is its inherent support for multi-document summarization. Because a single front-page teaser often summarizes a cluster of related articles on an internal page, the pipeline naturally produces complex data pairs that are typically difficult to acquire.
Datasets and Multilingual Evaluation
The primary output of this research is HEBTEASESUM, a new dataset comprising 7,774 samples from the Hebrew newspaper Hadashot (1984–1993). Notably, nearly half of these samples (3,869) are multi-document pairs.
To demonstrate the generalizability of this approach, the authors also curated a multilingual evaluation set. This set includes summary-article pairs from seven diverse languages: Norwegian, Icelandic, Estonian, Greek, Hebrew, Italian, and Polish.
Key Insights and Results
The study yielded several critical insights into the nature of journalistic summaries and the current state of LLM performance:
- Professional Quality and Abstractivity: Human experts rated the extracted teasers highly for coherence and consistency. Unlike "lead" sentences often used in automated scraping, these teasers are highly abstractive, featuring high "novel n-gram" ratios that indicate editors use original phrasing rather than simple extraction.
- Matching Performance: The zero-shot LLM approach achieved the highest accuracy for matching teasers to articles (95% precision). However, the TF-IDF method remained remarkably effective (83 F1-score), suggesting the pipeline is viable even for languages lacking advanced LLM support.
- The Coverage Gap: Evaluation of state-of-the-art models (DeepSeek-R1, Llama 3.3) revealed that while models are generally consistent (avoiding hallucinations), they struggle with "coverage"—failing to capture the same density of information found in the human-written teasers.
- Resource Disparity: The performance gap between models is significantly wider in low-resource languages compared to high-resource ones, reinforcing the need for localized, high-quality training data.
Limitations and Future Outlook
While promising, the method faces challenges related to historical archives. Optical Character Recognition (OCR) noise from old scans can degrade text quality, and segmentation errors—where the system fails to distinguish teaser text from surrounding visual elements—accounted for a majority of identification failures. Furthermore, the method is temporally bound to the era of modern newspaper layouts (20th century onwards).
Future work will focus on using these datasets to fine-tune models specifically for morphologically rich and low-resource languages, potentially bridging the performance gap in global NLP.
Summary updated