Learning Structured Representations of Entity Names using ActiveLearning and Weak Supervision

Kun Qian, Poornima Chozhiyath Raman, Yunyao Li, Lucian Popa

Introduction

In modern natural language processing, understanding the internal structure of entity names—such as parsing "Michael Jordan" into First Name and Last Name—is critical for tasks like entity normalization and variant generation. Traditionally, these tasks are handled by Entity Linking, which relies on matching names to entries in a Knowledge Base (KB) like Wikipedia using surrounding context.

However, significant challenges arise when names appear in isolation (e.g., search queries or database lists) or when dealing with specialized domains where no comprehensive KB exists. Teaching machines to recognize these internal patterns typically requires thousands of labeled examples or labor-intensive manual rule-writing. The PARTNER framework addresses these gaps by enabling machines to understand entity structures with minimal human intervention, achieving high accuracy using as few as 10 to 15 human-labeled examples.

The PARTNER Framework

The authors propose a hybrid framework that integrates semantic understanding with structural heuristics. The core model is a BERT-CRF architecture. It utilizes a BERT model to generate semantic embeddings of tokens, which are then processed through a Conditional Random Field (CRF) layer to predict label sequences.

The PARTNER architecture: Integrating BERT embeddings with structural features and a CRF layer for sequence labeling.

To enhance performance in low-resource settings, the model incorporates a "Structure Vector"—a 15-bit boolean vector describing the "shape" of a word (e.g., isAllCaps, containsNumber, isFirstLetterCapitalized). This allows the system to perform Weak Supervision via Structure: if the system knows "Apple Inc." is Name + Suffix, and it identifies that "Microsoft Corp." shares the same structure vector sequence, it can automatically propagate those labels.

Active Learning Strategy

Instead of labeling random data, PARTNER employs an active learning strategy based on an Informative Score. The system prompts users to label entities that are both representative (similar to many other unlabeled entities) and uncertain (where the model's confidence is low). This approach maximizes the utility of every human annotation, allowing the model to generalize from a handful of examples.

Experimental Evaluation

The framework was evaluated across four distinct datasets representing varying levels of complexity:

  • PER: 1,302 person names (labels: title, first, middle, last, suffix, degree).
  • ORG: 2,209 organization names (labels: corename, type, suffix, location).
  • DATE: 1,190 date expressions (labels: Year, MonthOfYear, Day).
  • LOG: 1,323 machine log entries (labels: host, time, filename, operation, etc.).

The LOG dataset is particularly challenging due to the length and complexity of the entities, which often cause traditional rule-based systems to fail.

Key Results

The PARTNER framework consistently outperformed baselines, including LUSTRE (a prior state-of-the-art active learning system) and standard CRF models.

Entity-level F1 score comparison on the PER dataset, demonstrating the efficiency of the proposed method.

Key findings include:

  • Minimal Effort: The framework reached F1-scores above 0.90 with only 7 to 13 user annotations per task.
  • High Precision: For specific components like First Name or Corename, the model achieved F1-scores as high as 0.989 and 0.996, respectively.
  • Robustness: While LUSTRE struggled with the complexity of the LOG dataset, PARTNER maintained high performance, proving that the synergy of semantic (BERT) and structural (Structure Vectors) features provides a robust "skeleton" for generalization.

Conclusion and Future Work

PARTNER demonstrates that high-quality entity parsing does not require massive datasets if active learning and structural weak supervision are combined effectively. Future research may focus on automated explanation generation, where the system uses structure vectors to explain its labeling logic to the user, further increasing transparency and trust in human-in-the-loop systems.

Summary updated

Abstract

Structured representations of entity names are useful for many entity-related tasks such as entity normalization and variant generation. Learning the implicit structured representations of entity names without context and external knowledge is particularly challenging. In this paper, we present a novel learning framework that combines active learning and weak supervision to solve this problem. Our experimental evaluation show that this framework enables the learning of high-quality models from merely a dozen or so labeled examples.
BibTeX
@inproceedings{qian2020learning,
  author = {Kun Qian and Poornima Chozhiyath Raman and Yunyao Li and Lucian Popa},
  title = {{Learning Structured Representations of Entity Names using ActiveLearning and Weak Supervision}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.517},
}
Figure