Abstract: Xinchi Chen, Chunchuan Lyu, Ivan Titov. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
2019.
Introduction
Semantic Role Labeling (SRL) is the task of identifying "who did what to whom" in a sentence. While modern systems are highly effective at local word-level encoding using LSTMs, they often struggle with global consistency. For example, a system might erroneously assign the "Subject" role to two different words for the same verb, even though most verbs only allow one. Traditional systems attempted to fix this with rigid "hard constraints," which are often too brittle for natural language. This paper introduces CapNetSRL, which uses Capsule Networks to allow different parts of a sentence to "negotiate" role assignments through a data-driven, iterative process.
The CapNetSRL Architecture
The core innovation of CapNetSRL is the application of Capsule Network logic to sentence structure.
- Word Capsules: Instead of a single vector, the model creates "word capsules" for every possible semantic role, encoding how well a word fits a specific role (e.g., is "injury" a likely "signer"?).
- Role Capsules & Dynamic Routing: Through multiple iterations of Dynamic Routing, the model refines connections between Word Capsules and "Role Capsules" (representing the overall proposition). If a word is a strong candidate for a role, its connection is strengthened.
- The Global Node: To capture high-level dependencies, the authors added a "Global Node." This node summarizes the entire proposition, allowing the model to recognize relationships between different roles (e.g., if a "Buyer" is found, the model should also look for a "Seller").
Iterative Refinement and Soft Constraints
A significant insight from this research is that explicitly forcing uniqueness via hard rules can actually hurt performance. Instead, the Dynamic Routing mechanism naturally learns "soft constraints." As the model iterates, it becomes "aware" of the global structure; if one word is identified as a strong subject, the routing mechanism suppresses the likelihood of other words taking that same role.
The authors found that the Global Node is essential for this refinement. Without it, the process acts primarily as a filter (increasing precision). With it, the model can actually "discover" missing arguments, significantly improving recall.
Key Results
CapNetSRL was evaluated on the CoNLL-2009 shared task datasets across seven languages, achieving state-of-the-art (SOTA) results on five of them (English, Chinese, Japanese, Czech, and Spanish).
- English Performance: The model reached an F1 score of 91.06% on the English test set, outperforming previous strong baselines.
- Robustness to Domain Shift: On out-of-domain data, the model showed a significant jump to 82.72% F1, suggesting that modeling structural interactions helps the model generalize better than memorizing word patterns.
- Efficiency: The iterative process is highly efficient, with peak performance typically reached in just 2 to 3 iterations.
Conclusion
By treating semantic roles as capsules and using dynamic routing, CapNetSRL effectively captures global structural constraints without relying on hand-coded rules. This approach proves particularly robust for out-of-domain text. Future work will explore adapting this architecture to span-based SRL and modeling interactions between multiple overlapping predicates simultaneously.
Summary updated