SSN_ARMM@ LT-EDI -ACL2022: Hope Speech Detection for Equality, Diversity, and Inclusion Using ALBERT model

Praveenkumar Vijayakumar, Prathyush S, Aravind P, Angel S, Rajalakshmi Sivanaiah, Sakaya Milton Rajendram, Mirnalinee T TACL 2022

TL;DR

This paper tackles the crucial task of detecting "hope speech" on social media to promote positive content and combat negativity. They leverage the ALBERT model, a transformer-based architecture, to achieve strong performance across multiple Dravidian languages and English, demonstrating its effectiveness in multilingual hope speech detection.

What Problem Does This Paper Address?

Social media has become a primary platform for human expression, but it also facilitates the spread of harmful content like hate speech. To counter this, there's a growing need for methods that can identify and amplify positive, constructive communication. This paper focuses on "hope speech detection," aiming to classify sentences as either "hope speech" or "non-hope speech." This is particularly challenging due to the prevalence of code-mixed language and class imbalances within datasets, especially for Dravidian languages. The goal is to build systems that can foster a more optimistic and inclusive online environment.

Methods

The authors employ a transformer-based approach for hope speech detection, specifically utilizing the ALBERT model. ALBERT is highlighted for its efficiency, requiring significantly fewer parameters than BERT while maintaining similar architectural capabilities. A key feature of ALBERT is its parameter-sharing mechanism across different layers, which contributes to its reduced size.

For tokenization, they use the ai4bharat/Indic-BERT model, which is pre-trained on 12 major Indian languages. This model is chosen for its ability to handle multilingual input and learn relationships between languages. The implementation involves tokenizing sentences with a maximum length of 400, enabling truncation. The authors also discuss the use of smart batches for efficient processing, sorting inputs by sequence length to minimize padding and preserve important tokens.

Transfer learning is applied by importing a pre-trained ALBERT model for sequence classification. The AdamW optimizer is used for weight updates, and a dynamic learning rate adjusted by a linear scheduler is employed. The training process involves calculating the average loss over batches.

Datasets

The study utilizes datasets provided by the Association for Computational Linguistics for the Hope Speech Detection for Equality, Diversity, and Inclusion (HopeEDI) competition. These datasets are multilingual and consist of YouTube comments manually labeled into two categories: "Hope-speech" and "Non-hope-speech." The languages covered are English, Tamil, Malayalam, and Kannada. The paper provides a breakdown of the number of comments in each language for training, testing, and development sets.

Key Insights and Ideas

  • ALBERT's Efficiency for Multilingual Tasks: The paper demonstrates that ALBERT, with its parameter-sharing and reduced parameter count, is a viable and effective model for multilingual hope speech detection, outperforming expectations in some languages.
  • Indic-BERT for Dravidian Languages: The choice of Indic-BERT for tokenization is crucial, as it's specifically designed to handle the complexities of multiple Indian languages, including Dravidian languages, which often feature code-mixing.
  • Smart Batching for Performance: The implementation of smart batching significantly reduces padding, leading to more efficient processing and potentially better utilization of computational resources.
  • The Importance of Hope Speech Detection: The work underscores the growing need to actively promote positive content online to foster a more inclusive and peaceful digital society.

Key Results

The model achieved impressive rankings in the competition:

  • 1st rank in Kannada with a weighted average F1 score of 0.750.
  • 2nd rank in Malayalam with a weighted average F1 score of 0.740.
  • 3rd rank in Tamil with a weighted average F1 score of 0.390.
  • 6th rank in English with a weighted average F1 score of 0.880.

The paper also provides precision and recall scores for each language, highlighting the model's performance across the board.

Limitations and Future Work

The authors acknowledge that multilingual comments can be challenging due to code-mixing, which can lead to misinterpretations and impact model results. They suggest that data augmentation could be a valuable technique to further fine-tune the model and improve its performance on more data in future work.

Summary updated

BibTeX
@inproceedings{vijayakumar2022ssn,
  author = {Praveenkumar Vijayakumar and Prathyush S and Aravind P and Angel S and Rajalakshmi Sivanaiah and Sakaya Milton Rajendram and Mirnalinee T T},
  title = {{SSN\_ARMM@ LT-EDI -ACL2022: Hope Speech Detection for Equality, Diversity, and Inclusion Using ALBERT model}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure