Abstract
This paper investigates the impact of various stemming strategies on Turkish Multi-Document Summarization (MDS). Using a newly created manually annotated dataset of news clusters, the study evaluates LexRank performance across raw text, morphological roots, derivational stems, and fixed-length truncation (prefixes). Results indicate that simple prefix-based truncation (Prefix-10) outperforms complex morphological analysis, suggesting that domain-specific vocabulary consistency in Turkish news clusters mitigates the need for sophisticated linguistic parsing.
Introduction
Multi-document summarization (MDS) aims to synthesize a collection of documents on a single topic into a concise, non-redundant summary. While well-studied in English, Turkish presents a unique "data sparseness" challenge due to its agglutinative nature. A single root word can take on a dizzying array of derivational and inflectional suffixes, changing its meaning or grammatical function.
In a standard English-style model, these variations look like entirely different terms, leading to a "vocabulary explosion." The number of unique terms in a Turkish corpus can be three times higher than in an English corpus of the same size, making it difficult for algorithms to identify semantic overlaps.
Methodology
The study utilizes LexRank, a graph-based summarization engine, as the primary evaluation framework. LexRank treats sentences as nodes in a graph and calculates centrality based on cosine similarity. The core experiment involves testing four distinct stemming policies to determine which best facilitates similarity detection:
- Raw: Words are used exactly as they appear in the text.
- Root: Every word is reduced to its most basic lemma (e.g., görüşmelerin → gör).
- Deriv: A middle-ground approach that retains suffixes changing the word's meaning (derivational) while removing those that only alter grammar (inflectional).
- Prefix-n: A computationally efficient method that truncates words to their first n characters.
Dataset Construction
Due to the lack of existing benchmarks for Turkish MDS, the authors developed a new dataset consisting of 21 news article clusters, with approximately 10 documents per cluster. Topics span politics, sports, and technology. Human annotators produced gold-standard summaries for each cluster, limited to 120 words. This dataset has been made publicly available to support future research in the field.
Analysis and Insights
The study reveals several key insights into the behavior of Turkish text in summarization tasks:
- The Homogeneity Hypothesis: Surprisingly, complex morphological analysis did not significantly outperform simpler methods. Within a specific topic cluster (e.g., news about a specific football match), journalists tend to use consistent inflected forms. This natural semantic narrowing dampens the "vocabulary explosion" effect.
- Over-Aggressive Rooting: The "Root" policy actually degraded performance. Stripping all suffixes removes semantic nuances that distinguish related but distinct concepts, leading to "false positive" similarities between sentences.
- The Efficiency of Truncation: Truncating words to 10 characters (Prefix-10) emerged as a "sweet spot." It captures the root and initial meaning-heavy suffixes without the overhead of a full morphological parser.
Results
Performance was evaluated using ROUGE scores, which measure n-gram overlap between system-generated and human-written summaries.
As shown in the results, Prefix-10 achieved the highest ROUGE-1 score of 0.438, followed by the Raw baseline at 0.428. Conversely, the Root approach performed the worst at 0.420. These findings suggest that for Turkish MDS, the computational cost of sophisticated linguistic tools may not be justified by the marginal performance gains.
Conclusion
This research demonstrates that for morphologically rich languages like Turkish, the most sophisticated linguistic tools are not always the most effective for extractive summarization. Future work should explore sentence compression techniques to further optimize summary density within word limits.
Summary updated