Abstract
Cross-lingual abstractive summarization (CLAS) is a complex task requiring models to generate concise summaries in a target language from source documents in a different language. Progress in this field has been hindered by the lack of high-quality, large-scale datasets. We present WikiLingua, a massive benchmark dataset featuring over 141,000 English articles from WikiHow aligned with parallel versions in 17 other languages. By utilizing images as language-agnostic anchors to align procedural steps, WikiLingua enables the development of direct cross-lingual summarization models that outperform traditional translation-heavy pipelines in both efficiency and quality.
Introduction
Abstractive summarization—generating a concise summary in a model's own words—is a significant challenge in natural language processing. Cross-lingual abstractive summarization (CLAS) adds another layer of complexity, as the source and target languages differ. Historically, CLAS research has relied on small-scale datasets or noisy social media data, often defaulting to two-step pipelines: translating the document then summarizing, or summarizing then translating. These pipelines are prone to error propagation and high latency, making them impractical for real-world applications.
WikiLingua addresses these issues by providing a high-quality, large-scale resource for training direct cross-lingual models. By leveraging the collaborative, multilingual nature of WikiHow, the authors have created a dataset that spans 18 languages, offering a robust foundation for future CLAS research.
Dataset Construction and Alignment
The core innovation in WikiLingua's construction is the use of images as "gold-standard" anchors. WikiHow articles describe procedural tasks across different languages, often using the same set of illustrative images to guide the reader through specific steps.
Because the textual order of steps might vary between cultures or translations, text-only alignment is often brittle. Images provide a universal, language-independent ground truth for identifying equivalent steps across different language versions of the same article. This allows for precise alignment of the detailed procedural text with its corresponding summary.
WikiLingua covers 18 languages, including Spanish, Russian, Turkish, and Vietnamese. It contains 141,457 unique English articles, with an average of 42,000 aligned articles per non-English language. This scale is orders of magnitude larger than previous datasets like MultiLing or Global Voices, which often contained only a few dozen articles per language.
Modeling Approach: Direct Cross-lingual Summarization
Moving away from computationally expensive two-step pipelines, the researchers focused on Direct Cross-lingual Summarization (DC). Using mBART, a large-scale multilingual transformer model, they explored three primary fine-tuning strategies:
- DC (Base): Direct fine-tuning on source-language articles and English summaries.
- DC+Synth: Incorporating synthetic data by translating English articles into target languages and pairing them with original English summaries to increase the training pool.
- DC+Synth+MT: A two-step process where the model is first trained on document-level machine translation (target language to English) before being fine-tuned for the summarization task.
Key Insights and Results
The study yielded several critical insights for the field of summarization:
- Challenging Lead Bias: Unlike news datasets where the most important information is usually at the beginning, WikiHow distributes information across multiple steps. This forces models to synthesize information from the entire document to produce a coherent summary.
- Synthetic Data Benefits: For lower-resource languages like Turkish or Vietnamese, adding synthetic data (DC+Synth) significantly boosted performance, making direct models competitive with or superior to translation-heavy baselines.
- Efficiency: Direct summarization eliminates the need for inference-time translation, making it faster and more cost-effective for production environments.
Quantitative evaluation using ROUGE scores showed that the DC+Synth+MT model consistently outperformed "translate-then-summarize" baselines. For instance, in Turkish-to-English tasks, the proposed method achieved a 42.76 ROUGE-1 score, compared to 31.57 for standard pipelines. Qualitative human evaluation confirmed that direct models maintain high fluency while capturing content more accurately than traditional methods.
Conclusion and Future Work
WikiLingua represents a major advancement in cross-lingual summarization resources. While the current work focuses on summarizing various languages into English, the dataset's structure supports future exploration into any-to-any summarization and multi-source summarization. By providing a large-scale, high-quality, and structurally diverse dataset, WikiLingua paves the way for more robust and efficient multilingual AI systems.
Summary updated