TL;DR
This paper introduces an interpretable machine learning model to predict online psychological help-seeking (OPHS) behavior during the COVID-19 pandemic. By combining temporal, topic, linguistic, and COVID-19 case data, the model accurately forecasts demand, offering crucial insights for resource allocation and intervention strategies.
What Problem Does This Paper Address?
The COVID-19 pandemic significantly impacted global mental health, leading to increased psychological distress and a surge in demand for mental health services. Traditional mental health systems struggled to cope, making online mental health services (OMHS) a critical alternative. However, accurately predicting the demand for these services was challenging, hindering effective resource allocation by governments and organizations. This paper addresses the need for an accurate, early, and interpretable prediction method for OPHS behavior, especially in the context of a public health crisis like COVID-19.
Methods
The researchers employed a multi-faceted approach:
- Data Crawling: They collected OPHS data from two large Chinese OMHS platforms ("One Psychology Community" and the MOE-CCNU OMHS platform) and COVID-19 case data from the National Health Commission.
- Feature Engineering:
- Neural Embedding-Based Topic Modeling: Used Word2vec to identify and categorize psychological problems and influential factors within OPHS texts, creating topic-based features.
- Linguistic Inquiry and Word Count (LIWC): Analyzed text for linguistic cues related to psychological states.
- Time Series Analysis (Prophet): Extracted temporal features (trend, seasonality) from OPHS numbers, including lagged values (1, 3, 7, 14 days).
- COVID-19 Case Data: Incorporated cumulative and new confirmed cases and deaths.
- Predictive Modeling:
- Random Forest (RF): Chosen as the primary machine learning algorithm due to its strong performance.
- Interpretable Machine Learning (SHAP): Shapley Additive Explanations were used to understand the contribution of each feature to the model's predictions, providing interpretability.
- Evaluation: The models were evaluated using Mean Absolute Error (MAE) and Pearson correlation coefficient, with 10-fold cross-validation.
Datasets
- "One Psychology Community" Data: Crawled 54,797 psychological help-seeking questions from January 31, 2018, to January 08, 2021.
- MOE-CCNU OMHS Platform Data: Collected daily OPHS numbers from January 31, 2020, to January 08, 2021, totaling 37,698 behaviors.
- National Health Commission Data: Official website for COVID-19 cases in China (cumulative and new confirmed cases and deaths).
Key Insights and Ideas
- OPHS as a Real-time Indicator: The study highlights OMHS platforms as valuable "always-on" data sources for tracking public mental health responses in real-time, especially during crises.
- Temporal Features are Key: The temporal dynamics of OPHS numbers (trends, past values) were found to be the most significant predictors, suggesting that past help-seeking behavior is a strong indicator of future behavior.
- Topics Trump General Language: While LIWC features provided some predictive power, the topic-based features derived from OPHS texts were more impactful and better suited for predicting OPHS, indicating that the specific content of help-seeking is more informative than general linguistic style.
- Cumulative COVID-19 Impact: The model revealed that cumulative COVID-19 cases and deaths, rather than daily new cases, significantly influenced OPHS behavior, suggesting that sustained societal impact is a stronger driver than immediate daily fluctuations.
- Interpretability is Crucial: The use of SHAP values demonstrated that machine learning models can be made understandable, allowing for actionable insights into why predictions are made, which is vital for policy and resource management.
Key Results
- Significant OPHS Increase: The COVID-19 pandemic led to a substantial increase in OPHS behavior across both platforms studied.
- Best Performing Model: The Random Forest model, incorporating temporal features of OPHS numbers, mental health topics, LIWC, and COVID-19 cases, achieved the best predictive performance.
- Feature Importance: Temporal features of OPHS numbers exhibited the strongest positive and negative predictive power. Topic features showed incremental improvements over LIWC features.
- COVID-19 Impact: Cumulative confirmed cases and deaths positively predicted OPHS behavior, while new cases/deaths did not show significant sensitivity. The impact of COVID-19 cases on OPHS grew with longer lag days.
- Predictive Horizon: The models could predict OPHS numbers with a lead time of up to two weeks.
Limitations and Future Work
- Data Limitations: The study was conducted with data from 2020 during the pandemic. Collecting more data over a longer period could improve model performance.
- Topic Feature Granularity: While topic features were effective, the authors suggest further exploration of their dimensions, as the OPHS number might change based on specific psychological problems rather than just the overall number of help-seekers.
- Model Comparison: While the proposed model outperformed Prophet for certain lag days, it did not achieve absolute advantages across all lag days.
- Future Directions: The authors suggest using deep learning forecasting methods to further improve prediction accuracy and exploring more granular topic features. They also emphasize the value of continuous monitoring of OMHS data for understanding public mental health dynamics.
Summary updated