TL;DR
This paper introduces MultiFED, a novel federated learning approach for detecting hate speech in low-resource Indian languages. By leveraging federated learning and a fair client selection strategy, MultiFED effectively addresses data scarcity and privacy concerns, achieving state-of-the-art performance across 13 Indian languages.
What Problem Does This Paper Address?
Social media has become a double-edged sword, facilitating communication while also becoming a breeding ground for hate speech. Detecting hate speech is crucial, but it's particularly challenging in Indian languages due to their linguistic diversity and cultural nuances. Traditional machine learning approaches often require centralizing sensitive user data, raising privacy concerns. Furthermore, existing research heavily favors well-resourced European languages, leaving low-resource Indian languages underserved. This paper tackles the dual challenge of limited data and privacy preservation in multilingual hate speech detection, specifically for Indian languages.
Methods
The core of this work is the MultiFED framework, which employs Federated Learning (FL). FL allows models to be trained locally on user devices, preventing sensitive data from leaving the user's control and thus preserving privacy. The paper proposes a novel fair selection approach for clients. This involves grouping clients based on lexical similarity of their local data, aiming to minimize model weight divergence and improve learning speed. This grouping strategy is designed to create more homogeneous client groups, which are then randomly selected for local training in each communication round. The paper also explores different aggregation algorithms like FedAvg, FedProx, and FedOpt, ultimately finding FedProx to be the most effective.
Datasets
The authors created a new multilingual dataset called multicomb by combining data from 13 Indian languages (Hindi, Tamil, Telugu, Kannada, Malayalam, Bengali, Marathi, Bhojpuri, Gujarati, Haryanvi, Odia, Punjabi) and English. These datasets were sourced from various platforms, including ShareChat, YouTube, Facebook, Twitter, and the TRAC-1 dataset. The paper details how these datasets were partitioned into IID (Independent and Identically Distributed) and non-IID (Non-Independent and Identically Distributed) client setups to simulate real-world scenarios with varying data distributions.
Key Insights and Ideas
- Federated Learning for Privacy and Performance: The study demonstrates that FL can not only protect user privacy but also achieve superior performance compared to centralized methods, especially in diverse data environments.
- Fair Client Selection is Crucial: Grouping clients based on lexical similarity and then randomly selecting from these groups significantly improves model generalization and prevents bias towards specific datasets. This addresses the challenge of data heterogeneity in FL.
- Domain Adaptation through Federated Learning: The federated approach effectively acts as a form of domain adaptation, allowing the model to leverage knowledge gained from one language or domain to improve performance on others, even with limited data.
- Importance of Pre-training and Continued Training: While pre-trained multilingual models like XLM-Roberta and mBERT show strong baseline performance, continued training (even in a federated setting) is vital for domain adaptation and achieving optimal results.
Key Results
MultiFED outperformed state-of-the-art baselines by approximately 8% in Accuracy and 12% in F-Score. The paper reports that the proposed MultiFED framework achieved a margin of 7.78% and 11.58% in accuracy and 9.59% and 11.91% in F-score over baselines in IID and non-IID settings, respectively. The study also found that performance generally increases with higher participation ratios (C) and more local training epochs. Data heterogeneity (controlled by alpha 'α') negatively impacts performance, with lower alpha values (more imbalance) leading to decreased accuracy, though F-scores sometimes improved with moderate imbalance.
Limitations and Future Work
The authors acknowledge that federated learning can face challenges with limited bandwidth, high latency, and resource-constrained user devices. They also note that the study was limited to 100 communication rounds due to computational constraints, suggesting that more iterations could potentially improve performance. Future work includes further personalization of client performance, improving the fairness of client selection, and reducing the resource consumption of each client. They also highlight that if client data is severely imbalanced in terms of labels and total size, federated learning might not outperform its centralized counterpart.
Summary updated