Introduction
Keyphrase Extraction (KPE) is a fundamental task in natural language processing aimed at identifying the most salient phrases within a document. While state-of-the-art models typically approach KPE as a supervised classification task, they often suffer from the "incomplete annotation" problem. In real-world datasets, human annotators frequently miss valid keyphrases. Under the standard "closed-world assumption," these unlabeled but valid phrases are treated as negative examples, leading to biased models that are penalized for correct predictions.
Proposed Methodology
To address these challenges, this research introduces a robust neural architecture combined with a novel training strategy based on negative sampling.
Neural Architecture
The model utilizes a RoBERTa sequence encoder to generate contextualized word embeddings. These embeddings are processed by a sliding CNN with multiple kernel sizes (1 to 5) to capture features for various n-grams. Finally, a binary classifier determines the likelihood of each candidate n-gram being a keyphrase.
Negative Sampling Strategy
The core innovation lies in the training process. Instead of treating all unlabeled phrases as negative instances, the authors employ negative sampling. By randomly selecting a small subset of unlabeled phrases as negatives in each training step, the model reduces the probability of repeatedly penalizing itself for identifying valid but unlabeled keyphrases. This approach also mitigates the extreme class imbalance inherent in KPE, where non-keyphrases vastly outnumber true keyphrases.
Experimental Results
The model was evaluated across scientific (Kp20k, Inspec, etc.) and open-domain (OpenKP) datasets.
Robustness to Incomplete Labels
To simulate varying degrees of annotation quality, the researchers created synthetic datasets by "masking" (removing) valid labels at rates ranging from 10% to 90%. As shown in the performance analysis, the negative sampling approach maintains significantly higher F1 scores compared to standard training as the masking rate increases.
Performance Highlights
- Open Domain: On the OpenKP dataset, the model achieved an F1@1 of 0.380, surpassing the previous state-of-the-art RoBERTa-Chunk model (0.355).
- Scientific Domain: The method remains competitive on well-annotated scientific benchmarks, indicating that negative sampling does not degrade performance even when labels are relatively complete.
Conclusion
By relaxing the closed-world assumption through negative sampling, this work provides a more resilient framework for keyphrase extraction in noisy, real-world environments. Future research will focus on modeling the dependencies between overlapping candidate phrases and implementing intelligent gating mechanisms for negative sample selection.
Summary updated