Deep Learning methods for Semantic Role Labeling in Indian Languages

Aishwary Gupta, Akshay Pawale, Manish ShrivastavaICON 2018 · OpenReview

Abstract

Semantic Role Labeling (SRL) identifies the predicate-argument structure of a sentence, answering "who did what to whom." While well-developed for English, SRL for Indian languages (ILs) like Hindi and Urdu has traditionally relied on manual feature engineering. This paper presents novel deep learning architectures, specifically syntax-aware BiLSTMs, that significantly improve SRL performance in ILs. By treating SRL as a sequence labeling task and incorporating minimal syntactic features, the proposed models achieve substantial gains over previous state-of-the-art systems.

Introduction

Semantic Role Labeling (SRL) is a core task in Natural Language Processing (NLP) that involves identifying predicates and their associated arguments (e.g., Agent, Patient, Location). For Indian languages, the primary challenge lies in the lack of large-scale annotated datasets and a historical reliance on "traditional" pipelines involving Support Vector Machines (SVMs) and manually engineered features. These systems are often brittle, as errors in upstream tools like dependency parsers propagate through the SRL system. This research investigates whether deep learning architectures can reduce this dependency and improve performance for Hindi and Urdu.

Methodology

The researchers proposed three distinct neural architectures to address these challenges:

  1. Model A (Path Embedding Model): An evolution of traditional methods that uses an LSTM to encode the dependency path between arguments and predicates into continuous vector representations.
  2. Model B (Syntax-Agnostic Model): A 2-layer Bidirectional LSTM (BiLSTM) that treats SRL as a sequence labeling task using IOB tagging, requiring only raw word tokens and a predicate indicator.
  3. Model C (Syntax-Aware Model): An extension of Model B that incorporates a single syntactic feature: the dependency label of each word.

To ensure grammatical consistency, the authors applied Constrained Decoding during the final prediction phase, enforcing rules such as the "Unique Core Roles" constraint (e.g., a verb cannot have two different subjects).

BiLSTM architecture for sequence labeling in SRL (Model B/C)

Datasets

The study utilized the Hindi and Urdu Propbanks, which are annotated at the "chunk" level—a standard for Indian language processing. The Hindi dataset contains 1,643 training sentences, while the Urdu dataset is larger with 4,657 training sentences. Notably, the Urdu dataset is entirely human-annotated, whereas the Hindi dataset contains some automatically generated annotations, which introduces additional noise.

Detailed statistics of the Hindi and Urdu Propbank datasets

Results and Key Insights

The transition to deep learning yielded significant performance improvements across both languages:

  • Performance Gains: In Hindi SRL, the F1-score rose from a baseline of 45.81 to 70.41. Urdu SRL also saw improvements, reaching an F1-score of 78.07.
  • Argument Identification: The models were particularly effective at identifying arguments (without labeling), achieving F1-scores of 93.32 for Hindi and 91.33 for Urdu.
  • The Power of Sequence Labeling: Treating SRL as a sequence labeling task (Models B and C) proved superior to the traditional "Identify then Classify" approach (Model A), as BiLSTMs better capture global sentence context.
  • Syntactic Boost: Adding even a single syntactic feature (dependency labels) provided a massive boost to the deep learning models, particularly in Hindi, helping to overcome data sparsity issues inherent in dependency paths.

Conclusion

This work demonstrates that deep learning can effectively handle the complexities of Indian languages even with limited annotated data. By reducing the reliance on heavy feature engineering, these models provide a more robust framework for IL NLP. Future work will focus on increasing dataset sizes, integrating morphological features (gender, number, case), and developing more sophisticated linguistic constraints for decoding to further refine performance.

Summary updated

BibTeX
@inproceedings{gupta2018deep,
  author = {Aishwary Gupta and Akshay Pawale and Manish Shrivastava},
  title = {{Deep Learning methods for Semantic Role Labeling in Indian Languages}},
  booktitle = {ICON},
  year = {2018},
  url = {https://openreview.net/forum?id=LnjoQatUtN},
}
Figure