Abstract
Semantic role labeling (SRL) involves extracting propositions (i.e., predicates and their typed arguments) from natural language sentences. State-of-the-art SRL models rely on powerful encoders (e.g., LSTMs) and do not model non-local interaction between arguments. We propose a new approach to modeling these interactions while maintaining efficient inference. Specifically, we use Capsule Networks: each proposition is encoded as a tuple of capsules, one capsule per argument type (i.e., role). These tuples serve as embeddings of entire propositions. In every network layer, the capsules interact with each other and with representations of words in the sentence. Each iteration results in updated proposition embeddings and updated predictions about the SRL structure. Our model substantially outperforms the non-refinement baseline model on all 7 CoNLL-2019 languages and achieves state-of-the-art results on 5 languages (including English) for dependency SRL. We analyze the types of mistakes corrected by the refinement procedure. For example, each role is typically (but not always) filled with at most one argument. Whereas enforcing this approximate constraint is not useful with the modern SRL system, the iterative procedure corrects the mistakes by capturing this intuition in a flexible and context-sensitive way.
Introduction
Semantic Role Labeling (SRL) is the task of determining "who did what to whom" in a sentence. Traditional neural SRL models rely on powerful encoders like Bi-LSTMs to process words, but they typically predict the role of each word independently. This independence assumption is problematic because language has inherent structural constraints; for instance, a single verb rarely has two different "agents." While older systems used rigid logical constraints to prevent such errors, modern neural systems often find these rules too inflexible. The central challenge is enabling a model to consider the entire proposition—the verb and all its potential arguments—simultaneously to make more coherent predictions.
Methodology: CapNetSRL
We propose CapNetSRL, which treats the SRL task as an iterative refinement process inspired by Capsule Networks.
Architecture
The model extends a standard Bi-LSTM encoder with two specialized capsule layers:
- Word Capsule Layer: Encodes argument-specific information for every word in the sentence.
- Role Capsule Layer: Represents the "slots" for the entire proposition (e.g., one capsule for the "Agent" role, one for "Patient," etc.).
Dynamic Routing and the Global Node
The core innovation is the use of the Capsule Network's dynamic routing algorithm. In each iteration, the model "routes" information from word capsules to role capsules. If multiple words compete for the same role, the routing mechanism facilitates a competition between them. To enhance expressiveness, we introduce a Global Node—a compressed representation of the entire proposition. This allows the model to capture interactions that are collaborative rather than just competitive, leveraging the holistic context of the sentence.
Experiments and Results
The model was evaluated on the CoNLL shared task datasets, covering seven languages (English, Chinese, Czech, Catalan, Spanish, German, and Japanese). We utilized ELMo embeddings for English and FastText for other languages.
Performance
CapNetSRL achieved state-of-the-art (SOTA) results on 5 out of 7 languages. On the English test set, it achieved an F1 score of 91.06%. The model also showed significant improvements on out-of-domain data, suggesting that modeling argument interactions improves generalization.
Analysis of Refinement
Our analysis confirms that the iterative refinement process is crucial for performance. The model's accuracy typically peaks after approximately two iterations of routing. Furthermore, the routing mechanism naturally discourages duplicate role assignments.
As illustrated in the figure above, the full CapNetSRL model (red line) effectively reduces the number of propositions with duplicate arguments over successive iterations. In contrast, the model without the Global Node (blue line) fails to maintain structural consistency, highlighting the Global Node's necessity for effective collaborative refinement.
Conclusion
CapNetSRL demonstrates that modeling argument interactions through Capsule Networks significantly improves SRL performance. By replacing independent role predictions with an iterative refinement process, the model learns to respect linguistic constraints like role uniqueness in a flexible, data-driven way. This approach achieves state-of-the-art results across multiple languages and provides a robust framework for capturing global structural dependencies in semantic analysis.
Summary updated