DLRG@LT-EDI-ACL2022:Detecting signs of Depression from Social Media using XGBoost Method

Herbert Sharen, Ratnavel RajalakshmiACL 2022

TL;DR

This paper tackles the crucial task of detecting depression from social media text, a growing concern due to the widespread use of these platforms. The researchers explored various machine learning models, ultimately finding that XGBoost achieved the highest accuracy in classifying depression levels, demonstrating its potential for early detection and intervention.

What Problem Does This Paper Address?

Depression is a significant global health issue, affecting millions and having detrimental impacts on personal well-being, family life, and professional environments. Early detection and treatment are vital for managing its effects. Traditional methods for diagnosing depression, such as clinical interviews and questionnaires, can be time-consuming and may not always be accessible. Social media, with its vast amount of user-generated text, presents a rich, albeit noisy, source of information that could potentially be leveraged for early depression detection. The challenge lies in accurately analyzing this text data to identify subtle signs of depression amidst everyday conversations and diverse emotional expressions.

Methods

The authors employed a multi-stage methodology for depression detection from social media text. The process began with data preprocessing, which involved cleaning and transforming the raw text data to minimize redundancies and ensure it was in a usable format. This included lemmatization, a Natural Language Processing (NLP) technique that reduces words to their base or dictionary form (lemma) to capture their semantic meaning more effectively. For feature extraction, they utilized the TF-IDF (Term Frequency-Inverse Document Frequency) method, which quantifies the importance of a word within a document relative to a corpus of documents.

To classify the depression levels, three machine learning models were implemented and compared:

  • Support Vector Machine (SVM): A powerful classification algorithm that finds an optimal hyperplane to separate data points into different classes.
  • Random Forest (RF): An ensemble method that builds multiple decision trees and aggregates their predictions to improve accuracy and reduce overfitting.
  • XGBoost Classifier: An advanced gradient boosting algorithm known for its speed and accuracy, which is an optimized version of gradient boosting.

The system was designed to classify posts into three categories: "not depressed," "moderately depressed," and "severely depressed."

Datasets

The study utilized the CodaLa dataset, which contains social media postings in English. This dataset comprises 8891 texts, each labeled with one of the three depression categories. The paper also references a dataset provided by organizers (Sampath et al.) for the shared task on detecting signs of depression from social media text.

Key Insights and Ideas

  • Social media as a proxy for mental health: The paper reinforces the idea that social media text can serve as a valuable, albeit complex, indicator of an individual's mental state, offering a potential avenue for early detection of conditions like depression.
  • XGBoost's superior performance: The consistent finding across the paper is that XGBoost significantly outperformed both SVM and Random Forest in classifying depression levels, highlighting its effectiveness for this specific task.
  • The importance of text preprocessing: Techniques like lemmatization and TF-IDF are crucial for transforming unstructured text data into a format that machine learning models can effectively learn from, underscoring the foundational role of NLP in this domain.
  • Multi-class classification is feasible: The successful implementation of a three-class classification system (not depressed, moderately depressed, severely depressed) demonstrates the potential for nuanced depression detection beyond a simple binary classification.

Key Results

The study found that the XGBoost classifier achieved the highest classification accuracy, reaching 0.643 with a Macro F1-score of 0.54. Random Forest achieved an accuracy of 0.564, and SVM achieved an accuracy of 0.567. The paper also reports weighted F1-scores, recall, and precision for each model in Table 1, with XGBoost generally showing the best performance across these metrics as well.

Limitations and Future Work

Limitations: * The paper mentions that the dataset collected from CodaLabs consists of 8891 texts, and the shape of the training set was (8891, 1500) while the testing set was (3245, 1500). This implies that the original dataset might have had more features or that some preprocessing steps resulted in this dimensionality. The exact nature of the 1500 features is not explicitly detailed, beyond the mention of TF-IDF. * The accuracy scores, while the highest achieved by the authors, are still relatively modest (around 64%), suggesting room for improvement in model performance.

Future Work: * The authors suggest that parameter adjustment is required to fully leverage the advantages of the XGBoost model over competing methods, indicating that hyperparameter tuning could further enhance performance. * While not explicitly stated as future work, the modest accuracy scores suggest that exploring more advanced NLP techniques, different feature engineering approaches, or larger, more diverse datasets could be beneficial.

Summary updated

BibTeX
@inproceedings{sharen2022dlrg,
  author = {Herbert Sharen and Ratnavel Rajalakshmi},
  title = {{DLRG@LT-EDI-ACL2022:Detecting signs of Depression from Social Media using XGBoost Method}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure