Abstract
Automatic keyword extraction (AKE) is a critical component of modern information retrieval and natural language processing, facilitating tasks such as text summarization, topic analysis, and document indexing. Despite its importance, many existing AKE methods struggle to balance statistical frequency with linguistic and semantic relevance. This paper presents a universal, post-processing-based approach designed to enhance the performance of any AKE method. By integrating Part-of-Speech (PoS) tagging, domain-specific thesauri, and Wikipedia-based named entity recognition, the proposed framework re-evaluates candidate keywords to ensure they align with human-centric selection patterns. Experimental results across 17 datasets and five state-of-the-art (SOTA) AKE methods demonstrate significant and consistent performance gains, with an average F1-score improvement of 25.8%.
Introduction
As digital textual data continues to grow exponentially, the ability to automatically extract meaningful keywords has become indispensable. However, AKE remains a challenging task because there is no universal consensus on what constitutes a "keyword." Current SOTA methods often fall into two categories: purely statistical models that ignore linguistic structure, and models that lack semantic awareness, failing to recognize technical concepts or established entities.
This research addresses these gaps by introducing a "plug-and-play" post-processor. This framework does not require retraining existing models; instead, it refines their output by applying linguistic filters and external knowledge bases to better mimic human keyword selection.
Methodology
The proposed approach consists of a three-step post-processing pipeline that re-evaluates the scores of candidate keywords generated by a baseline AKE method:
1. PoS-Tagging Filter
Analysis of human-selected keywords reveals that over 90% follow specific linguistic patterns, primarily consisting of nouns and noun phrases. The post-processor utilizes PoS-tagging to identify the grammatical structure of candidates. If a candidate does not fit these established linguistic profiles (e.g., it is a lone verb or adverb), its score is "zeroed out," effectively filtering out noise that statistical methods often misidentify as significant.
2. Context-Aware Thesauri
To incorporate domain-specific knowledge, the system checks if a candidate keyword appears in specialized thesauri, such as MeSH (Medical Subject Headings) or AGROVOC (Agriculture). When a match is found, the candidate's importance score is doubled. To make this process scalable, the researchers implemented an Automatic Context Identification step, using a machine-learning classifier to automatically select the most relevant thesaurus for a given document.
3. Wikipedia Named Entities
Wikipedia serves as a massive, context-independent repository of human-defined concepts. The framework treats Wikipedia page titles as a proxy for "keyword-worthiness." Candidates matching Wikipedia entries (excluding common stop-words) receive a score boost, ensuring that recognized entities are prioritized in the final extraction list.
Experimental Setup
The framework was evaluated using 17 diverse English-language datasets, including scientific abstracts (Inspec, PubMed), news (KPCrowd), and miscellaneous benchmarks (SemEval 2010 & 2017). To test the universality of the approach, it was applied to five distinct SOTA baseline algorithms:
- YAKE!
- KP-Miner
- RaKUn
- LexRank
- SIFRank+
Key Insights and Discussion
The Wikipedia Overlap
A significant finding of the study was that 64.39% of human-selected "gold standard" keywords are Wikipedia page titles. This high correlation justifies the use of Wikipedia as a powerful, general-purpose filter for identifying high-quality keywords across various domains.
Linguistic Constraints
The research confirmed that human keyword selection is highly constrained by length and grammar. Over 82% of human-selected keywords are unigrams or bigrams, and 94.7% are three words or fewer. By enforcing these length constraints and PoS rules, the post-processor successfully eliminated "junk" phrases often picked up by language-agnostic statistical models like YAKE!.
Universal Compatibility
Because the method only requires a list of candidates and their initial scores, it can be "bolted on" to any AKE method. This makes it a highly flexible tool for researchers and practitioners who wish to improve existing pipelines without the computational cost of retraining models.
Results
The experimental results were consistently positive across all tested configurations:
- Significant F1 Gains: The combined approach improved the F1-score of baseline methods by an average of 25.8%.
- Algorithm-Specific Success: The most dramatic improvement was observed with RaKUn, which saw performance increases of up to 53.8%.
- Consistency: In some configurations, the method improved performance in 100% of the cases tested, demonstrating its robustness across different datasets and document types.
- Automation Viability: The automatic context classifier performed nearly as well as manual thesaurus selection, proving the system's readiness for unsupervised, large-scale applications.
Conclusion and Future Work
This paper demonstrates that a simple, semantic-aware post-processing framework can significantly enhance the accuracy of Automatic Keyword Extraction. While currently focused on English, the logic of the approach is language-independent and could be adapted to other languages by updating PoS patterns and thesauri. Future research will explore more sophisticated Word Sense Disambiguation (WSD) to handle polysemy, ensuring that score boosts are applied only when the context of a word matches its definition in external knowledge bases.
Summary updated