Introduction
Semantic typing—the process of identifying entities, relations, and events—is fundamental to structural language understanding. Traditionally, models treat labels as arbitrary indices, ignoring their inherent semantic meaning and struggling with "closed-world" limitations where they cannot predict types unseen during training.
Methods
The authors introduce UNIST (Unified Semantic Typing), a framework that treats semantic typing as a matching problem within a joint embedding space.
Joint Semantic Embedding
Using a pre-trained RoBERTa encoder, UNIST projects both input sentences and candidate labels into the same high-dimensional space. During training, a Margin Ranking Loss is employed to pull correct label embeddings closer to the sentence representation while pushing incorrect ones away.
Unified Prompting
To handle diverse tasks (entity, relation, and event typing) within a single architecture, UNIST utilizes natural language task descriptions as prompts. This allows the model to leverage its pre-trained knowledge to understand the specific objective of the query.
Datasets and Evaluation
UNIST was evaluated across several benchmarks:
- UFET (Ultra-Fine Entity Typing): Over 2,500 free-form labels.
- TACRED: Standard relation classification (41 types).
- MAVEN: Event typing in the general domain (168 types).
- FewRel: Zero-shot relation classification testing.
Key Insights
- Labels as Data: By encoding the actual text of labels (e.g., "bodily harm"), the model can generalize to rare or unseen categories using its linguistic priors.
- Multi-Task Synergy: Training on multiple tasks simultaneously improves cross-task knowledge transfer and overall robustness.
Results
UNIST-Large achieved a new state-of-the-art F1 score of 75.5 on the TACRED relation classification task. Furthermore, it demonstrated exceptional zero-shot capabilities, outperforming previous models on UFET's unseen labels by 7.2% and reaching 91.2% accuracy on the FewRel 5-way 0-shot benchmark.
Limitations
The framework currently assumes that "spans of interest" are pre-identified and its performance is sensitive to the descriptiveness of label names. Future work may involve integrating mention detection and scaling to even more diverse datasets.
Summary updated