Abstract
Semantic Role Labeling (SRL) is a critical task in Natural Language Processing, yet most state-of-the-art models are hyper-optimized for English and require significant manual effort to adapt to other languages. This paper introduces DAMESRL, a modular, open-source framework designed to make deep SRL accessible and customizable across diverse linguistic structures. By enabling a "plug-and-play" approach to neural components, DAMESRL allows researchers to tailor architectures to specific language traits and data constraints. Our evaluation across English, German, and Arabic demonstrates that while self-attention excels in high-resource settings, recurrent models and character-level embeddings are vital for low-resource and morphologically rich languages.
Introduction
Semantic Role Labeling (SRL) aims to identify the predicate-argument structure of a sentence, answering "who did what to whom, where, and when." While deep learning has reduced the need for manual feature engineering, the "one-size-fits-all" approach often fails across different languages. English relies heavily on word order, whereas languages like German and Arabic utilize complex inflections to signal grammatical roles. Furthermore, many languages lack the massive annotated datasets available for English. DAMESRL addresses these challenges by providing a flexible framework to experiment with various neural architectures.
The DAMESRL Framework
The authors propose a four-phase modular pipeline for constructing SRL models, allowing for high degrees of customization.
- Phase I: Word Representation: Users can combine standard word embeddings with predicate indicators and character-level Bi-LSTM representations. Character-level features are particularly effective for capturing the internal structure (prefixes/suffixes) of words in inflected languages.
- Phase II: Sequence Representation: This "engine" layer supports standard Bi-LSTMs, Highway Bi-LSTMs for deeper gradient flow, or modern Self-Attention (Transformer-style) networks.
- Phase III: Output Modeling: Hidden states are converted into probabilities using either Softmax or Conditional Random Fields (CRF), the latter being useful for capturing dependencies between adjacent labels.
- Phase IV: Inference: The framework offers a choice between fast Greedy decoding and globally optimal Viterbi decoding.
Beyond model construction, DAMESRL includes visualization tools that generate intuitive HTML "grids" to represent semantic roles.
Experimental Setup and Datasets
The framework was evaluated on three datasets representing different linguistic families and resource levels:
- English (CoNLL’05): A large dataset characterized by strict word order.
- German (CoNLL’09): A medium-sized dataset representing a morphologically rich language.
- Arabic (CoNLL’12): A small, highly inflected dataset representing a low-resource scenario.
Key Insights and Results
The researchers compared five configurations: HLstm, Char, CRFm, Att, and CharAtt. The CharAtt model (Character embeddings + Self-Attention) emerged as the top performer, achieving F1 scores of 84.9 (English), 73.5 (German), and 55.2 (Arabic).
Several critical insights were gained:
- Linguistic Nuance: Character-level embeddings significantly improved performance for German and Arabic by providing sub-word cues essential for understanding inflections.
- Data Sensitivity: While self-attention models were approximately 5x faster to train and highly effective on large datasets, they performed poorly on small datasets (e.g., 2,000 sentences).
- Recurrent Bias: In low-resource scenarios, LSTMs and CRFs outperformed attention-based models. The inherent sequential bias of recurrent models helps them learn sentence structure more effectively when training data is scarce.
Conclusion
DAMESRL provides a robust platform for end-to-end deep SRL, offering competitive results on English and some of the first deep learning benchmarks for German and Arabic without manual linguistic features. Future work will explore integrating pre-trained language models like BERT to improve performance in low-resource settings and further mitigate the data-hunger of attention-based modules.
Summary updated