Prediction of Online Psychological Help-Seeking Behavior During the COVID-19 Pandemic: An Interpretable Machine Learning Method

Hui Liu, Lin Zhang, Weijun Wang, Yinghui Huang, Li Shen, Zhihong Ren, Zongkui ZhouFrontiers in Public Health 2022 · DOI

TL;DR

This paper introduces an interpretable machine learning model to predict online psychological help-seeking (OPHS) behavior during the COVID-19 pandemic. By combining temporal, topic, linguistic, and COVID-19 case data, the model accurately forecasts demand, offering crucial insights for resource allocation and intervention strategies.

What Problem Does This Paper Address?

The COVID-19 pandemic significantly impacted global mental health, leading to increased psychological distress and a surge in demand for mental health services. Traditional mental health systems struggled to cope, making online mental health services (OMHS) a critical alternative. However, accurately predicting the demand for these services was challenging, hindering effective resource allocation by governments and organizations. This paper addresses the need for an accurate, early, and interpretable prediction method for OPHS behavior, especially in the context of a public health crisis like COVID-19.

Methods

The researchers employed a multi-faceted approach:

  • Data Crawling: They collected OPHS data from two large Chinese OMHS platforms ("One Psychology Community" and the MOE-CCNU OMHS platform) and COVID-19 case data from the National Health Commission.
  • Feature Engineering:
    • Neural Embedding-Based Topic Modeling: Used Word2vec to identify and categorize psychological problems and influential factors within OPHS texts, creating topic-based features.
  • Linguistic Inquiry and Word Count (LIWC): Analyzed text for linguistic cues related to psychological states.
  • Time Series Analysis (Prophet): Extracted temporal features (trend, seasonality) from OPHS numbers, including lagged values (1, 3, 7, 14 days).
  • COVID-19 Case Data: Incorporated cumulative and new confirmed cases and deaths.
  • Predictive Modeling:
    • Random Forest (RF): Chosen as the primary machine learning algorithm due to its strong performance.
  • Interpretable Machine Learning (SHAP): Shapley Additive Explanations were used to understand the contribution of each feature to the model's predictions, providing interpretability.
  • Evaluation: The models were evaluated using Mean Absolute Error (MAE) and Pearson correlation coefficient, with 10-fold cross-validation.

Datasets

  • "One Psychology Community" Data: Crawled 54,797 psychological help-seeking questions from January 31, 2018, to January 08, 2021.
  • MOE-CCNU OMHS Platform Data: Collected daily OPHS numbers from January 31, 2020, to January 08, 2021, totaling 37,698 behaviors.
  • National Health Commission Data: Official website for COVID-19 cases in China (cumulative and new confirmed cases and deaths).

Key Insights and Ideas

  • OPHS as a Real-time Indicator: The study highlights OMHS platforms as valuable "always-on" data sources for tracking public mental health responses in real-time, especially during crises.
  • Temporal Features are Key: The temporal dynamics of OPHS numbers (trends, past values) were found to be the most significant predictors, suggesting that past help-seeking behavior is a strong indicator of future behavior.
  • Topics Trump General Language: While LIWC features provided some predictive power, the topic-based features derived from OPHS texts were more impactful and better suited for predicting OPHS, indicating that the specific content of help-seeking is more informative than general linguistic style.
  • Cumulative COVID-19 Impact: The model revealed that cumulative COVID-19 cases and deaths, rather than daily new cases, significantly influenced OPHS behavior, suggesting that sustained societal impact is a stronger driver than immediate daily fluctuations.
  • Interpretability is Crucial: The use of SHAP values demonstrated that machine learning models can be made understandable, allowing for actionable insights into why predictions are made, which is vital for policy and resource management.

Key Results

  • Significant OPHS Increase: The COVID-19 pandemic led to a substantial increase in OPHS behavior across both platforms studied.
  • Best Performing Model: The Random Forest model, incorporating temporal features of OPHS numbers, mental health topics, LIWC, and COVID-19 cases, achieved the best predictive performance.
  • Feature Importance: Temporal features of OPHS numbers exhibited the strongest positive and negative predictive power. Topic features showed incremental improvements over LIWC features.
  • COVID-19 Impact: Cumulative confirmed cases and deaths positively predicted OPHS behavior, while new cases/deaths did not show significant sensitivity. The impact of COVID-19 cases on OPHS grew with longer lag days.
  • Predictive Horizon: The models could predict OPHS numbers with a lead time of up to two weeks.

Limitations and Future Work

  • Data Limitations: The study was conducted with data from 2020 during the pandemic. Collecting more data over a longer period could improve model performance.
  • Topic Feature Granularity: While topic features were effective, the authors suggest further exploration of their dimensions, as the OPHS number might change based on specific psychological problems rather than just the overall number of help-seekers.
  • Model Comparison: While the proposed model outperformed Prophet for certain lag days, it did not achieve absolute advantages across all lag days.
  • Future Directions: The authors suggest using deep learning forecasting methods to further improve prediction accuracy and exploring more granular topic features. They also emphasize the value of continuous monitoring of OMHS data for understanding public mental health dynamics.

Summary updated

Abstract

Online mental health service (OMHS) has been named as the best psychological assistance measure during the COVID-19 pandemic. An interpretable, accurate, and early prediction for the demand of OMHS is crucial to local governments and organizations which need to allocate and make the decision in mental health resources. The present study aimed to investigate the influence of the COVID-19 pandemic on the online psychological help-seeking (OPHS) behavior in the OMHS, then propose a machine learning model to predict and interpret the OPHS number in advance. The data was crawled from two Chinese OMHS platforms. Linguistic inquiry and word count (LIWC), neural embedding-based topic modeling, and time series analysis were utilized to build time series feature sets with lagging one, three, seven, and 14 days. Correlation analysis was used to examine the impact of COVID-19 on OPHS behaviors across different OMHS platforms. Machine learning algorithms and Shapley additive explanation (SHAP) were used to build the prediction. The result showed that the massive growth of OPHS behavior during the COVID-19 pandemic was a common phenomenon. The predictive model based on random forest (RF) and feature sets containing temporal features of the OPHS number, mental health topics, LIWC, and COVID-19 cases achieved the best performance. Temporal features of the OPHS number showed the biggest positive and negative predictive power. The topic features had incremental effects on performance of the prediction across different lag days and were more suitable for OPHS prediction compared to the LIWC features. The interpretable model showed that the increase in the OPHS behaviors was impacted by the cumulative confirmed cases and cumulative deaths, while it was not sensitive in the new confirmed cases or new deaths. The present study was the first to predict the demand for OMHS using machine learning during the COVID-19 pandemic. This study suggests an interpretable machine learning method that can facilitate quick, early, and interpretable prediction of the OPHS behavior and to support the operational decision-making; it also demonstrated the power of utilizing the OMHS platforms as an always-on data source to obtain a high-resolution timeline and real-time prediction of the psychological response of the online public.
BibTeX
@inproceedings{liu2022prediction,
  author = {Hui Liu and Lin Zhang and Weijun Wang and Yinghui Huang and Li Shen and Zhihong Ren and Zongkui Zhou},
  title = {{Prediction of Online Psychological Help-Seeking Behavior During the COVID-19 Pandemic: An Interpretable Machine Learning Method}},
  booktitle = {Frontiers in Public Health},
  year = {2022},
  doi = {10.3389/fpubh.2022.814366},
}
Figure