Abstract
Semantic Role Labeling (SRL) has traditionally relied on structured decoders to identify argument spans. This paper investigates whether these complex decoders still provide value when paired with powerful pre-trained encoders like BERT. By comparing BIO-tagging, span-based, and two-step (head-selection) methods across multiple datasets, the authors demonstrate that BERT acts as a "leveler," largely neutralizing the advantages of structured decoders. While structured methods excel with static embeddings by reducing boundary and attachment errors, BERT's internal representations already address these issues, making simpler decoders like BIO-tagging highly competitive and more efficient.
Introduction
Semantic Role Labeling (SRL) is a core NLP task that involves identifying predicates and their associated arguments to determine "who did what to whom." This task is fundamentally a span extraction problem. While recent advancements in SRL have been driven by sophisticated encoders like Transformers and BERT, the choice of the "decoder"—the component responsible for extracting argument spans—has received less scrutiny. This research evaluates whether the structural priors of complex decoders remain necessary in the era of large-scale pre-training.
Methods
The study compares three primary decoding strategies:
- BIO-tagging: A sequence labeling approach where each token is tagged as Beginning (B), Inside (I), or Outside (O) of an argument span.
- Span-based: A method that considers all possible word combinations (up to a fixed length) and classifies them as arguments.
- Two-step (Head-Selection): A strategy that first identifies a "head" word for an argument and then determines its boundaries. This includes "HeadSyntax," which uses external syntactic trees, and "HeadAuto," which learns heads automatically.
These models were evaluated on industry-standard PropBank datasets, including CoNLL-2005 and CoNLL-2012, as well as cross-lingual transfer settings.
Results and Analysis
The experiments reveal a "BERT Leveler" effect. With static embeddings (e.g., fastText), structured decoders significantly outperform BIO-tagging. However, when BERT is used as the encoder, the performance gap narrows to less than 0.4 F1 points on the CoNLL-2005 test set. For instance, the simplest BIO-tagger achieved 87.58 F1 compared to 87.93 F1 for the most complex Head-based tagger.
Error analysis (Figure 6) shows that structured decoders primarily reduce "Boundary" and "Attachment" errors compared to BIO-tagging when using static embeddings. However, BERT is inherently proficient at resolving these same error types. Consequently, the structural benefits of complex decoders become redundant when a high-capacity encoder is present.
Efficiency remains a key differentiator. BIO-tagging is the fastest method, processing over 700 sequences per second, whereas span-based models are nearly twice as slow (355.8 sequences per second) due to their quadratic complexity.
Conclusion
The choice of decoder for SRL is less critical when using powerful encoders like BERT, as the encoder captures much of the structural information previously provided by the decoder. While BIO-tagging offers the best balance of speed and performance for most modern applications, structured decoders like HeadSyntax remain valuable in low-resource settings where external syntactic signals can supplement weaker embeddings.
Summary updated