Abstract
Instead of using expensive manual annotations, researchers have proposed to train named entity recognition (NER) systems using heuristic labeling rules. However, devising labeling rules is challenging because it often requires a considerable amount of manual effort and domain expertise. To alleviate this problem, we propose GLaRA, a graph-based labeling rule augmentation framework, to learn new labeling rules from unlabeled data. We first create a graph with nodes representing candidate rules extracted from unlabeled data. Then, we design a new graph neural network to augment labeling rules by exploring the semantic relations between rules. We finally apply the augmented rules on unlabeled data to generate weak labels and train a NER model using the weakly labeled data. We evaluate our method on three NER datasets and find that we can achieve an average improvement of +20% F1 score over the best baseline when given a small set of seed rules.
Introduction
Named Entity Recognition (NER)—the task of identifying entities like "diseases" or "chemicals" in text—usually requires massive amounts of manually labeled data. To circumvent this "annotation bottleneck," weak supervision has emerged as a viable alternative, utilizing heuristic labeling rules (e.g., "if a word ends in -noma, label it a Disease") created by domain experts. However, manually crafting these rules remains a labor-intensive process requiring significant domain knowledge. GLaRA addresses this by taking a minimal set of human-written "seed" rules and automatically discovering hundreds of similar, effective rules from unlabeled data.
Methodology
The GLaRA framework operates through a multi-stage pipeline designed to transition from a small set of human-defined rules to a robust, high-performance NER model.
1. Candidate Rule Extraction and Graph Construction
GLaRA first extracts a diverse set of candidate rules from unlabeled text, including surface forms, morphological features (prefixes/suffixes), contextual n-grams, and dependency relations. These candidates form the nodes of a graph. Edges are established between nodes based on the semantic similarity of the entities they match, calculated using contextual embeddings (e.g., ELMo).
2. Rule Augmentation via Graph Neural Networks
The core of GLaRA is a Graph Attention Network (GAT) that propagates labeling confidence from the initial seed rules to unlabeled candidate nodes. To enhance the separation between entity and non-entity rules, the authors implement a novel "class distance-based loss" ($L_{dist}$), which maximizes the distance between the centroids of positive and negative rule clusters in the embedding space.
3. Model Training
The highest-ranked augmented rules are applied to unlabeled data to generate weak labels. A generative model, such as LinkedHMM, is employed to resolve conflicts between overlapping or contradictory rules. Finally, the resulting noisily labeled dataset is used to train a high-performance discriminative NER model, such as a BiLSTM-CRF.
Experimental Evaluation
The framework was evaluated on three benchmark datasets: NCBI (biomedical/disease), BC5CDR (biomedical/disease and chemical), and LaptopReview (aspect term extraction).
Key Results
GLaRA demonstrates significant performance improvements, particularly in low-resource settings. With only 10 seed rules, GLaRA achieved an average improvement of +22 F1 points over the strongest baseline.
As shown in the results for the NCBI dataset, GLaRA consistently outperforms the baseline across varying numbers of seed rules, reaching an F1 score of 80.2—approaching the performance of fully supervised models (87.2 F1) without requiring any manual token-level annotations.
Discussion and Conclusion
The success of GLaRA hinges on its ability to leverage semantic relatedness and explicitly utilize negative seed rules to define entity boundaries. By maximizing the distance between positive and negative rule clusters, the system effectively filters high-quality rules from noisy candidates.
While currently limited to predefined rule templates, future iterations of GLaRA could explore more complex logical patterns and sophisticated rule representations. Nonetheless, GLaRA represents a significant step toward bridging the gap between expert-driven rule creation and automated data-driven learning, enabling high-quality NER with minimal human intervention.
Summary updated