Multi-Target Cross-Lingual Summarization: a novel task and a language-neutral approach

Diogo Pernes, Gonçalo M. Correia, Afonso MendesEMNLP 2024 · DOI

Introduction

Cross-lingual summarization (CLS) has traditionally focused on generating a summary in a single target language from a source document in another language. However, real-world applications often require "one-to-many" summaries—for instance, providing news updates to a global audience in multiple languages simultaneously. This paper introduces Multi-Target Cross-Lingual Summarization (MTXLS), a task that demands not only high-quality individual summaries but also semantic coherence across all target languages.

The MTXLS Task

The primary challenge in MTXLS is the "trustworthiness" gap. Independent generation for each language often leads to inconsistent content, where one summary might be faithful while another contains hallucinations. As shown in Figure 2, while traditional methods like Summarize-and-Translate (S&T) ensure coherence by translating a single pivot summary, they suffer from pivot bias and error propagation. MTXLS (Figure 2c) explicitly models the dependencies between summaries in different languages to ensure they convey the same information regardless of the target language.

Graphical models for cross-lingual summarization: (a) Summarize-and-Translate, (b) Direct CLS, and (c) the proposed Multi-Target CLS (MTXLS) with inter-summary dependencies.

Neutral Re-Ranking (NeutralRR)

To address the computational complexity of selecting the most coherent set of summaries from multiple candidates, the authors propose NeutralRR. This framework operates in three stages:

  1. Candidate Generation: A multilingual model (e.g., mT5) generates $k$ candidate summaries for each target language.
  2. Semantic Scoring: The system utilizes SONAR embeddings—fixed-size vectors that represent meaning across languages—to calculate pairwise similarity between candidates.
  3. Heuristic Search: Since finding the optimal set is NP-hard, the authors employ a dynamic programming heuristic. By randomly permuting the target languages and finding the most similar "path" of summaries across the chain, they efficiently approximate the maximum coherence.

Crucially, NeutralRR is "language-neutral," treating all languages equally without relying on an English pivot, which reduces bias and improves performance for non-English language pairs.

Experimental Results

The approach was evaluated on a clustered version of the CrossSum dataset across seven major languages. The researchers used standard ROUGE scores alongside Machine Translation Quality Estimation (QE) metrics like CometKiwi and BLASER 2.0 to measure coherence.

Performance metrics across varying candidate counts (k). NeutralRR shows significant gains in coherence (CometKiwi) while maintaining stable ROUGE-2 scores.

As illustrated in Figure 3, the proposed NeutralRR method significantly outperforms the many-to-many baseline in semantic coherence. A key finding is the behavior as the number of candidates ($k$) increases: CometKiwi scores improve substantially, indicating that larger search spaces allow the re-ranker to find more consistent sets of summaries. Notably, this gain in coherence comes with only a marginal impact on individual summary quality (ROUGE-2), suggesting that the most consistent summaries are often among the most faithful.

Conclusion

MTXLS represents a significant step toward reliable multilingual information systems. By prioritizing semantic coherence through a language-neutral re-ranking framework, NeutralRR bridges the gap between translation accuracy and summary faithfulness. Future work will explore integrating these coherence constraints directly into model training to reduce the computational overhead of post-processing re-ranking.

Summary updated

BibTeX
@inproceedings{pernes2024multi,
  author = {Diogo Pernes and Gonçalo M. Correia and Afonso Mendes},
  title = {{Multi-Target Cross-Lingual Summarization: a novel task and a language-neutral approach}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.755},
}
Figure