Abstract
We present SetExpander, a corpus-based system for expanding a seed set of terms into a more complete set of terms that belong to the same semantic class. SetExpander implements an iterative end-to-end workflow. It enables users to easily select a seed set of terms, expand it, view the expanded set, validate it, re-expand the validated set and store it, thus simplifying the extraction of domain-specific fine-grained semantic classes. SetExpander has been used successfully in real-life use cases including integration into an automated recruitment system and an issues and defects resolution system.
Introduction
In Natural Language Processing (NLP), Term Set Expansion (TSE) is the task of taking a small set of "seed" terms (e.g., "Siri" and "Cortana") and identifying others that belong to the same semantic group (e.g., "Alexa" or "Google Assistant").
A primary challenge in TSE is distinguishing between topical similarity and functional similarity. Standard "bag-of-words" models often capture topical relations; for instance, the seed "Python" might yield "bytecode" or "compiler." However, for set expansion, functional similarity—identifying other programming languages like "Java" or "C++"—is typically desired. Furthermore, systems must handle varying levels of granularity, such as distinguishing between general colors and specific citrus fruits based on the provided seeds.
Methodology
SetExpander addresses these challenges through a multi-context embedding approach and an iterative human-in-the-loop workflow.
Multi-Context Embeddings
Rather than relying on a single word vector, SetExpander trains five distinct types of embedding models to capture different semantic nuances:
- Linear Bag-of-Words: The standard approach analyzing neighboring words.
- Explicit Lists: Captures terms appearing in structured lists (e.g., "A, B, and C").
- Syntactic Dependency (Dep): Analyzes the grammatical relationship between terms and other sentence elements.
- Symmetric Patterns (SP): Identifies terms in specific linguistic patterns like "X and Y."
- Unary Patterns (UP): Focuses on specific n-grams where the term serves as a placeholder.
These contexts are combined using a Multilayer Perceptron (MLP) classifier, which processes similarity scores across these models to predict if a candidate term belongs in the target set.
Iterative Workflow
The system is designed for iterative refinement. Users select initial seeds, review the system's expansion, validate correct terms, and re-expand the set. This process allows the user to steer the granularity of the results dynamically, solving ambiguity problems by allowing human validation to define the intended semantic class.
Evaluation and Results
SetExpander was evaluated across three diverse datasets to demonstrate its versatility:
- English Wikipedia: Used for general-purpose evaluation, achieving a Mean Average Precision (MAP@10) of 0.83.
- Recruitment Corpus: Applied to resume and job description data to generate skill lists for specific roles. Precision scores for the top 100 candidates ranged from 70.5% to 98.0%.
- Software Defect Repository: Used to identify duplicate issues in a massive technical database. The system improved duplicate detection precision by over 10% and reduced list generation time from several weeks to just a few hours.
Conclusion
The success of SetExpander highlights the importance of functional similarity over topical similarity for TSE tasks. By leveraging complementary context types and an iterative user interface, the system effectively handles fine-grained semantic classes across various domains. Future work may involve integrating transformer-based embeddings to further enhance the multi-context framework.
Summary updated