Glocal: Incorporating Global Information in Local Convolution for Keyphrase Extraction

Animesh Prasad, Min-Yen KanNAACL 2019 · OpenReview

Abstract

Keyphrase extraction is a critical task in natural language processing that involves identifying the most salient terms in a document. While Graph Neural Networks (GNNs) have shown promise in modeling local word relationships, they often lack a global perspective on word importance. This paper introduces Glocal, a model that integrates global centrality scores from unsupervised algorithms like TextRank into the Graph Attention Network (GAT) architecture. By scaling local attention weights with global importance, Glocal achieves state-of-the-art performance on full-text scientific article datasets.

Introduction: The Local-Global Dilemma

Keyphrase extraction requires balancing local context (immediate word neighborhoods) with global importance (document-wide centrality). Traditional Graph Convolutional Networks (GCNs) are inherently "near-sighted." Because they normalize node influence relative to immediate neighbors, a globally insignificant word in a sparse neighborhood can receive the same weight as a critical word in a dense one. This lack of global ranking information hinders their ability to identify keyphrases, which are by definition globally salient within a document.

The Glocal Architecture

The authors propose the Glocal model (Global + Local) to address this limitation by modifying the Graph Attention Network (GAT) mechanism.

  1. Baseline GAT: Standard GATs learn the relative importance of neighbors through an attention mechanism ($\alpha$). However, this remains a local computation.
  2. Global Injection: The model first computes TextRank scores ($\beta$) for every word in the document graph. TextRank is an unsupervised random walk algorithm that captures global centrality.
  3. Glocal Convolution: Instead of relying solely on learned local attention, Glocal scales neighbor features by the product of local attention and the global TextRank score ($\beta \times \alpha$).
  4. Non-Normalization: A key innovation is the decision not to normalize the global $\beta$ scores. This allows globally important nodes to exert a stronger influence during training, effectively anchoring the local convolutions to the document's most significant parts.
Comparison of GCN, GAT, and Glocal Convolution architectures.

The final pipeline uses a 2-layer Glocal network, followed by post-processing to reconstruct multi-token phrases and re-rank them based on individual token probabilities.

Experimental Setup

The model was evaluated on three full-text scientific article datasets, which are significantly more challenging than abstract-only datasets due to increased noise and scale:

  • Schutz (2008): 1,323 documents.
  • Krapivin (2009): 2,304 documents.
  • SemEval (2010): 1,095 documents.

Key Results and Performance

Glocal consistently outperformed traditional sequence models and standard graph architectures across all benchmarks.

Performance results (F1 scores) across three datasets for various models.
  • State-of-the-Art Results: Glocal achieved a significant improvement over CopyRNN, a strong sequence-to-sequence baseline, with a 2% absolute F1 increase on the Schutz dataset.
  • Graph Superiority: The model demonstrated a substantial lead over standard GCNs and GATs. For instance, on the SemEval dataset, Glocal reached an F1@15 of 33.5, compared to 25.9 for GAT and 21.4 for GCN.
  • Robustness: Evaluation showed that Glocal performs well on both author-assigned and reader-assigned keywords, indicating its ability to capture diverse definitions of "importance."

Insights: Why Glocal Works

  • Synergy over Features: Simply adding TextRank scores as input features to a GAT yielded negligible improvements (<0.5%). However, using these scores to scale the convolution led to a ~5% gain, suggesting that global information is most effective when it modulates feature flow.
  • Normalization Trade-offs: While normalization is standard for stability, the authors found that normalizing global scores within local neighborhoods erases the very global context they intended to capture.
  • Implicit Length Bias: By aggregating features from neighbors, high-weight nodes spread their influence to adjacent tokens, naturally favoring the identification of coherent multi-word sequences.

Conclusion and Future Work

Glocal demonstrates that integrating unsupervised global centrality into supervised local convolutions significantly enhances keyphrase extraction. Future research could explore more complex graph constructions—incorporating section headers or citation graphs—and apply the "Glocal" principle to other graph-based tasks like community detection or social network analysis.

Summary updated

BibTeX
@inproceedings{prasad2019glocal,
  author = {Animesh Prasad and Min-Yen Kan},
  title = {{Glocal: Incorporating Global Information in Local Convolution for Keyphrase Extraction}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
  url = {https://openreview.net/forum?id=r1-cHQ-dZB},
}
Figure