TL;DR
This paper presents the first automatic extractive summarization system for the Punjabi language. It addresses the challenge of summarizing Punjabi text by developing a novel system that preprocesses text, identifies relevant sentences based on statistical and linguistic features, and generates a concise summary.
What Problem Does This Paper Address?
Text summarization aims to condense a source document into a shorter form while retaining its essential information and meaning. While automatic summarization systems exist for many languages, Punjabi, an official language of India with a significant number of speakers, has historically lacked such tools. This paper tackles the problem of creating an effective extractive summarization system for Punjabi, which involves identifying and extracting the most important sentences from a Punjabi document to form a summary. This is particularly challenging due to the unique linguistic characteristics of Punjabi, such as spelling variations and the need for language-specific processing.
Methods
The proposed system operates in two main phases: Pre-Processing and Processing.
Pre-Processing Phase: * Punjabi Words and Sentences Boundary Identification: Crucial for segmenting the text correctly. * Punjabi Stop Words Elimination: Removes common words that don't carry significant meaning, improving the focus on relevant content. * Punjabi Language Stemmer for Nouns and Proper Names: Reduces words to their root form, aiding in feature extraction and normalization. This is a novel contribution for Punjabi. * Allowing Input Restrictions: Ensures the input text adheres to certain criteria (e.g., predominantly Gurmukhi script). * Elimination of Duplicate Sentences: Prevents redundancy in the final summary. * Normalization of Punjabi Nouns: Addresses the issue of non-standardized spellings in Punjabi by normalizing variations.
Processing Phase: This phase calculates scores for each sentence to determine its importance. Key features used include:
- Punjabi Sentence Relative Length Feature: Shorter sentences are generally avoided, while longer ones might contain more information.
- Punjabi Keywords/Title Keywords Identification: Identifies thematic words using the TF-ISF (Term Frequency-Inverse Sentence Frequency) approach.
- Numeric Data Identification: Sentences containing numerical data are considered important.
- Punjabi Named Entity Recognition (NER): A rule-based system developed to identify proper nouns, a novel contribution for Punjabi.
- Punjabi Sentence Headlines and Next Lines Identification: Headlines and the lines immediately following them are given high importance.
- Punjabi Nouns and Proper Names Identification: Sentences containing nouns and proper names are weighted higher.
- Punjabi Cue Phrase Identification: Sentences starting with or containing specific cue phrases (e.g., "in conclusion") are identified as important.
Finally, a feature-weight equation is used to calculate the overall score for each sentence, and the top-ranked sentences are selected for the summary.
Datasets
- Punjabi Corpus: Used for creating frequency lists of stop words and for analyzing unique words. The paper mentions a corpus of around 11.29 million words and 2.03 lakh unique words.
- Punjabi News Documents: Fifty Punjabi news documents were used for testing, totaling 6185 sentences and 72689 words. These were randomly taken from the Punjabi Ajit news corpus.
- Punjabi Stories: Fifty Punjabi stories were used for testing, totaling 17538 sentences and 178400 words. These were randomly taken from www.likhari.org.
- Gold Summaries: Manually produced summaries by three human experts were used for evaluation at 10%, 30%, and 50% compression ratios.
Key Insights and Ideas
- Pioneering Punjabi NLP Resources: A significant contribution is the development of several crucial lexical resources for Punjabi NLP from scratch, including a Punjabi stemmer, noun normalizer, proper names list, stop words list, etc., as no prior work existed in these areas.
- Feature Importance Varies by Document Type: The system performs exceptionally well on news documents at lower compression ratios (10%) because headlines and lead sentences capture the core information. However, for stories, higher compression ratios (30-50%) are needed for satisfactory summarization, as stories lack the structured headline format.
- Extractive Summarization is Feasible for Punjabi: The paper demonstrates that even with the linguistic complexities of Punjabi, an extractive summarization system can achieve high accuracy.
- NER and Stemming are Crucial: The development of novel Punjabi-specific NER and stemming techniques proved vital for effective feature extraction and sentence scoring.
Key Results
The paper reports both intrinsic and extrinsic evaluation results:
Intrinsic Evaluation (using F-Score and Cosine Similarity): * Punjabi News Documents: * Average F-Score: 97.87% (10%), 95.32% (30%), 94.63% (50%) * Average Cosine Similarity: 0.9814 (10%), 0.9629 (30%), 0.9522 (50%) * Punjabi Stories: * Average F-Score: 81.78% (10%), 89.32% (30%), 94.21% (50%) * Average Cosine Similarity: 0.8226 (10%), 0.8838 (30%), 0.9432 (50%)
Extrinsic Evaluation (Question Answering and Keyword Association Tasks): * Punjabi News Documents: * Question Answering Accuracy: 78.95% (10%), 81.38% (30%), 88.75% (50%) * Keyword Association Accuracy: 80.13% (10%), 92.37% (30%), 96.32% (50%) * Punjabi Stories: * Question Answering Accuracy: 80.65% (10%), 84.26% (30%), 90.72% (50%) * Keyword Association Accuracy: 84.29% (10%), 90.68% (30%), 95.16% (50%)
Overall, the system performs well, especially at 50% compression ratio, where the summary is comprehensive enough to describe the whole text for both news documents and stories.
Limitations and Future Work
Limitations: * The paper notes that for Punjabi stories, the system's performance is not as strong at 10% compression because stories lack the explicit headline structure found in news, leading to less informative summaries at low compression. * The question answering task accuracy for news documents is lower at 10% compression because the extracted headlines and next lines are not sufficient to answer all questions.
Future Work: * The authors suggest that the developed lexical resources (stemmer, normalizer, etc.) can be beneficial for developing other Natural Language Processing applications for Punjabi. * Further research could explore abstractive summarization techniques for Punjabi. * Investigating more advanced feature selection and weight learning methods could further enhance performance.
Summary updated