Abstract
Instead of using expensive manual annotations, researchers have proposed to train named entity recognition (NER) systems using heuristic labeling rules. However, devising labeling rules is challenging because it often requires a considerable amount of manual effort and domain expertise. To alleviate this problem, we propose GLARA, a graph-based labeling rule augmentation framework, to learn new labeling rules from unlabeled data. We first create a graph with nodes representing candidate rules extracted from unlabeled data. Then, we design a new graph neural network to augment labeling rules by exploring the semantic relations between rules. We finally apply the augmented rules on unlabeled data to generate weak labels and train a NER model using the weakly labeled data. We evaluate our method on three NER datasets and find that we can achieve an average improvement of +20% F1 score over the best baseline when given a small set of seed rules.
Introduction
Training robust Named Entity Recognition (NER) models typically necessitates vast quantities of manually labeled data—a process that is both time-consuming and costly. Weak supervision offers a promising alternative by utilizing heuristic labeling rules (e.g., "if a word ends in -noma, label it as a Disease"). However, manually crafting these rules remains a significant bottleneck, requiring deep domain expertise and extensive effort to achieve sufficient coverage.
GLaRA (Graph-based Labeling Rule Augmentation) addresses this challenge by automating the expansion of a small set of "seed" rules. By representing rules as nodes in a graph and leveraging Graph Neural Networks (GNNs), GLaRA discovers hundreds of accurate new rules from unlabeled data, significantly boosting performance in low-resource settings.
Methodology
The GLaRA framework operates through a multi-stage pipeline designed to transition from a few manual seeds to a comprehensive set of weak labels.
1. Rule Extraction and Graph Construction
The system extracts candidate rules from an unlabeled corpus based on six templates: SurfaceForm, Prefix, Suffix, PreNgram, PostNgram, and DependencyRule. These candidates form the nodes of a graph. To establish edges, GLaRA computes rule embeddings by averaging the contextual representations (using ELMo) of all entity mentions triggered by a given rule. Edges are then drawn between rules that match semantically similar contexts.
2. Rule Augmentation via GNN
Rule discovery is treated as a semi-supervised node classification task. Using a Graph Attention Network (GAT), GLaRA propagates confidence scores from the initial seed rules (both positive and negative) across the graph. A specialized distance-based loss function ($L_{dist}$) is employed to maximize the separation between positive (entity-matching) and negative (non-entity) rules in the embedding space, ensuring a sharp decision boundary.
3. Denoising and Model Training
The augmented rules generate a "label matrix" from the unlabeled data. To resolve conflicts and noise among overlapping rules, a generative model (LinkedHMM) estimates the latent "true" labels. These denoised weak labels are then used to train a final BiLSTM-CRF discriminative model.
Experimental Evaluation
GLaRA was evaluated on three benchmarks: NCBI (biomedical diseases), BC5CDR (diseases and chemicals), and LaptopReview (technical aspect terms).
Key Results
The framework demonstrates substantial improvements over traditional weak supervision methods, particularly when the number of initial seed rules is small.
- Low-Resource Gains: With only 10 seed rules per type, GLaRA outperformed the LinkedHMM baseline by an average of 22 F1 points.
- Expert-Level Performance: On the NCBI dataset, GLaRA achieved an F1 score of 80.2, rivaling specialized systems like SwellShark that require significantly more manual tuning.
- Ablation Study: The $L_{dist}$ loss function proved critical, consistently improving performance by effectively separating positive and negative rule clusters.
Discussion and Conclusion
GLaRA's success stems from its ability to leverage the semantic context of matched entities to find related rules. By treating rules as nodes in a graph, it automates the discovery of domain-specific features (e.g., identifying prefixes like mesot- or dyst- in medical texts) that non-experts might overlook.
While GLaRA currently relies on predefined rule templates and high-quality contextual embeddings, it represents a major step toward reducing the human effort required for NER. Future work may explore more complex rule types and document-level consistency to further enhance the robustness of automated rule discovery.
Summary updated