TL;DR
This paper tackles the detection of patronizing and condescending language (PCL) on social media, a subtle form of discrimination often overlooked. The Amrita_CEN team employed several machine learning models, finding that logistic regression, enhanced with SMOTE for class imbalance, achieved the best performance.
What Problem Does This Paper Address?
Social media platforms, while connecting people, can also foster hostile environments through subtle forms of discrimination. While hate speech and abusive language have received significant research attention, patronizing and condescending language (PCL) – language that conveys a pompous attitude or elicits pity – has been relatively neglected. This type of language can contribute to exclusion and inequity, particularly for vulnerable groups. The paper addresses the challenge of computationally detecting PCL in social media text.
Methods
The Amrita_CEN team implemented a multi-stage machine learning pipeline.
- Preprocessing: This involved cleaning the text by removing stopwords, URLs, and special characters, followed by tokenization and lemmatization to reduce vocabulary and sentence vector dimensions.
- TF-IDF Vectorization: Textual data was converted into numerical vectors using Term Frequency-Inverse Document Frequency (TF-IDF).
- Feature Engineering: To combat the class imbalance inherent in PCL datasets, the team employed SMOTE (Synthetic Minority Over-sampling Technique). SMOTE synthetically generates data for minority classes by creating new instances based on existing ones.
- Machine Learning Modeling: Seven machine learning models were explored: three classical models (Logistic Regression, SVM, Decision Tree) and four ensemble models (Bagging, Random Forest, Gradient Boosting, XGBoost). Hyperparameter tuning was performed for logistic regression.
Datasets
The paper utilizes datasets from SemEval-2022 Task 4, which is divided into two subtasks:
- Subtask 1: A binary classification task to predict whether a given paragraph contains condescension. The dataset consists of 10,469 paragraphs describing vulnerable social categories.
- Subtask 2: A multi-label classification task to predict if a paragraph falls under one of the top 7 PCL taxonomies (e.g., Unbalanced power relations, Shallow solution, Compassion). This dataset contains 993 paragraphs.
Key Insights and Ideas
- PCL as a Neglected Area: The paper highlights that PCL, despite its potential to harm vulnerable communities, has received less research attention compared to other forms of harmful language.
- SMOTE's Effectiveness for Imbalance: The use of SMOTE proved crucial in addressing the class imbalance problem, a common challenge in PCL detection, allowing minority classes to have a more significant impact on model training.
- Logistic Regression's Robustness: Despite exploring more complex ensemble methods, logistic regression emerged as a surprisingly effective and efficient model for PCL detection.
Key Results
- Best Performing Model: Logistic Regression outperformed all other tested models, achieving the highest macro average F1 score for both subtasks.
- Subtask 1 Performance: For Subtask 1, the team achieved an F1 score of 0.39 on the test dataset and ranked 60th in the competition.
- Subtask 2 Performance: For Subtask 2, they achieved a macro average F1 score of 0.45.
- Efficiency: Logistic regression also demonstrated a lower execution time compared to ensemble models.
Limitations and Future Work
- Limitations: The paper does not explicitly detail limitations beyond the inherent challenges of class imbalance and the relatively neglected nature of PCL research.
- Future Work: The authors suggest that future work could explore deep learning and BERT-based approaches, which they believe may yield better results than classical machine learning models for PCL detection.
Summary updated