TL;DR
This paper introduces a novel big data architecture for real-time prediction of suicidal ideation from social media. By combining batch processing for model training with real-time streaming analysis, the system aims to identify at-risk individuals earlier, potentially saving lives.
What Problem Does This Paper Address?
Suicidal ideation is a critical public health issue, and its prevalence is increasing. While social media offers a platform for individuals to express their emotions, the sheer volume and real-time nature of this data make traditional analysis methods insufficient for early detection. This paper addresses the challenge of effectively processing massive, high-velocity social media streams to identify potential suicidal ideation in real-time, thereby enabling timely intervention.
Methods
The proposed methodology employs a two-phase approach: batch processing for model development and real-time streaming prediction.
Batch Processing Phase: * Data Collection: Historical data was collected from Reddit subreddits like "Suicide Watch" and "Teenagers Forum." * Preprocessing: Standard NLP techniques were applied, including filtering duplicates, URLs, punctuation, and special symbols, followed by case folding, tokenization, stopword removal, and lemmatization. * Feature Engineering: Various combinations of N-gram, TF-IDF, and Count Vectorizer were explored to extract optimal features. * Model Development: Six Spark ML algorithms were trained and evaluated: Naïve Bayes (NB), Logistic Regression (LR), Linear Support Vector Classifier (LinearSVC), Decision Tree (DT), Random Forest (RF), and Multilayer Perceptron (MLP). * Model Selection: The best-performing model from the batch phase was chosen for real-time prediction.
Real-Time Streaming Prediction Phase: * Data Ingestion: Twitter API (using Tweepy) was used to collect real-time tweets based on keywords related to suicidal ideation. * Stream Processing: Apache Kafka was used for data ingestion and buffering, while Apache Spark Structured Streaming processed the incoming tweet streams. * Prediction: The pre-trained best model from the batch phase was applied to the processed real-time data to predict suicidal ideation. * Visualization: Results were pushed to a Kafka output topic and visualized using Power BI.
The core novelty lies in the integration of a robust big data architecture (Spark and Kafka) for handling the scale and velocity of social media data, coupled with a systematic evaluation of multiple feature engineering and classification techniques for this specific task.
Datasets
- Batch Dataset: Collected from Reddit subreddits "Suicide Watch" and "Teenagers Forum" via the Pushshift API. The dataset comprised approximately 232,074 posts, with 116,028 classified as suicidal and 116,037 as non-suicidal. After cleaning, it resulted in 232,042 rows.
- Real-Time Streaming Dataset: Tweets collected from Twitter API using keywords like "feel," "want to die," and "kill myself." A total of 764 tweets were collected for the real-time phase.
The paper does not introduce a new benchmark dataset but utilizes publicly available data from Reddit and real-time data from Twitter.
Key Insights and Ideas
- Social Media as a Window to Mental Health: The study reinforces the idea that social media platforms are valuable sources for understanding public sentiment and detecting mental health risks like suicidal ideation, especially due to the anonymity they offer.
- Big Data Architecture is Crucial: Traditional methods are insufficient for analyzing the volume and speed of social media data. The paper demonstrates the necessity and effectiveness of a big data architecture (Spark and Kafka) for this task.
- Feature Engineering Matters: The combination of feature extraction techniques significantly impacts model performance. The study found that (Unigram + Bigram) + CV-IDF was a highly effective combination.
- MLP's Superiority: The Multilayer Perceptron (MLP) classifier consistently outperformed other models across various feature extraction methods, highlighting its suitability for this complex classification problem.
- Real-time Prediction is Feasible: The successful implementation of a real-time streaming prediction pipeline demonstrates the potential for immediate identification and intervention.
Key Results
- Batch Processing: The Multilayer Perceptron (MLP) classifier achieved the highest accuracy of 93.47% with the (Unigram + Bigram) + CV-IDF feature combination in the batch processing phase. Logistic Regression (LR) followed with 92.14% accuracy.
- Real-Time Prediction: The system successfully processed real-time tweets, predicting suicidal ideation. The results showed that 9.29% of the analyzed tweets were predicted as suicide, while 90.71% were non-suicide.
- Feature Performance: The (Unigram + Bigram) + CV-IDF feature extraction combination generally yielded better results compared to other combinations.
- Model Comparison: MLP consistently showed superior performance across different feature sets, followed by LR. DT and RF underperformed in this study.
Limitations and Future Work
Limitations: * The study acknowledges that the real-time dataset was relatively small (764 tweets). * The paper mentions that the MLP classifier with (Unigram + CV-IDF) achieved a high accuracy of 93.33%, which is slightly lower than the best result.
Future Work: * Increase Feature Set: Incorporate additional features like emoticons, special characters, and symbols to improve optimal feature extraction and reduce misclassification. * Expand Dataset: Gather textual data from other social media platforms to create a more representative and varied dataset. * Multilingual Support: Extend the system to handle multiple languages, such as Turkish and Arabic. * Advanced Models: Explore Ensemble LSTM and CNN models for better performance, especially for sequential information and local feature engineering. * User Interface: Develop a web or mobile interface to serve as a text-analysis tool for detecting individuals' health status.
Summary updated