Analyzing Digital Evidence From a Telemental Health Platform to Assess Complex Psychological Responses to the COVID-19 Pandemic: Content Analysis of Text Messages (Preprint)

Thomas D. Hull, Jacob Levine, Niels Bantilan, Angel N. Desai, Maimuna S. Majumder2020 · DOI

Introduction

The COVID-19 pandemic has left an indelible mark on global mental health, yet traditional methods for assessing this burden often fall short. While standard clinical tools like the GAD-7 (General Anxiety Disorder-7) and PHQ-9 (Patient Health Questionnaire-9) are effective for broad screening, they are relatively blunt instruments. They were designed to capture broad categories of disorder, not the nuanced, multi-faceted psychological fallout specific to a global health crisis—such as health paranoia, obsessive cleaning, or the unique grief associated with social distancing.

This study addresses this "blind spot" by leveraging natural language processing (NLP) to analyze unstructured text data from a large-scale telemental health platform. By examining the actual words patients use in therapy, researchers aimed to identify the specific "symptom signatures" of the COVID-19 era and differentiate this burden from pre-pandemic baselines.

Methodology and Data

The researchers utilized a massive, de-identified dataset from Talkspace, a popular digital mental health service. The study analyzed 219,156 transcripts from 169,889 unique patients between January 1, 2017, and June 9, 2020. The demographic profile was predominantly female (73.2%) and young (55% aged 26-35).

The "digital phenotyping" pipeline involved several sophisticated steps:

  1. Baseline Establishment: Longitudinal data from 2017–2020 provided a pre-pandemic baseline for intake severity using GAD-7 and PHQ-9 scores.
  2. Keyword Isolation: Messages containing pandemic-related "seed words" (e.g., virus, quarantine, corona) were identified.
  3. NLP Analysis: Using Term Frequency-Inverse Document Frequency (TF-IDF), the team tracked word frequency changes over time and correlated these trajectories with COVID-19 mentions.
  4. Thematic Categorization: Terms were grouped into themes using Empath, a tool that uses word embeddings to categorize text into functional categories like "health," "government," or "negative emotion."
  5. Clinical Mapping: Finally, these themes were mapped to formal ICD-10 (International Classification of Diseases) diagnostic categories to understand the clinical implications of the pandemic-related language.

Key Findings: The "COVID-19 Syndrome"

The analysis revealed a distinct psychological profile for pandemic-related distress, characterized by acute anxiety rather than depression.

Anxiety vs. Depression

The study found a significant spike in intake anxiety scores—a 1.42 point increase on the GAD-7 during the peak of the first wave (mid-March to early April 2020). However, there was no detectable change in intake depression scores. This suggests the initial psychological response was primarily one of acute threat—a "fight-or-flight" reaction—rather than the lethargy or hopelessness typically associated with clinical depression.

Beyond Standard Metrics

The NLP analysis identified 24 distinct symptoms that fall outside standard anxiety and depression criteria. These included:

  • Hypochondriasis: Intense worry about unexplained physical aches and pains.
  • Paranoia: A heightened sense of needing to be constantly on guard.
  • Obsessive-Compulsive Behaviors: A drive to perform repetitive acts (e.g., cleaning or checking) to mitigate perceived threats.
  • Acute Stress and PTSD: Symptoms such as nightmares, avoidance, and hypervigilance mapping to ICD-10 categories F43.0 and F43.1.

Interestingly, the word-stem "lone" (lonely, loneliness) was not significantly correlated with pandemic concerns in this dataset. The authors hypothesize that patients were either too overwhelmed by immediate health and economic threats to process social isolation as "loneliness," or that the digital nature of the platform itself provided a mitigating sense of connectivity.

Implications and Limitations

The results support a "dimensional" approach to psychiatry, aligning with frameworks like the NIMH’s Research Domain Criteria (RDoC). Instead of forcing patients into rigid diagnostic boxes, this approach views symptoms across a spectrum, allowing for more personalized and accurate treatment.

However, the study has limitations. The data comes from a private, paid platform, which may not represent populations with lower socioeconomic status or those without digital access. Furthermore, the sample was demographically skewed toward young, white women, which is typical for telemedicine but limits broader generalizability.

Conclusion

By mapping the digital phenotype of COVID-19 distress, this research demonstrates that the pandemic's mental health impact is far more complex than traditional screeners suggest. Monitoring these newly identified symptoms alongside standard anxiety and depression metrics is crucial for developing specialized treatment protocols as the long-term effects of the pandemic continue to unfold.

Summary updated

Abstract

<sec> <title>BACKGROUND</title> The novel COVID-19 disease has negatively impacted mortality, economic conditions, and mental health. These impacts are likely to continue after the COVID-19 pandemic ends. There are no methods for characterizing the mental health burden of the COVID-19 pandemic, and differentiating this burden from that of the prepandemic era. Accurate illness detection methods are critical for facilitating pandemic-related treatment and preventing the worsening of symptoms. </sec> <sec> <title>OBJECTIVE</title> We aimed to identify major themes and symptom clusters in the SMS text messages that patients send to therapists. We assessed patients who were seeking treatment for pandemic-related distress on Talkspace, which is a popular telemental health platform. </sec> <sec> <title>METHODS</title> We used a machine learning algorithm to identify patients’ pandemic-related concerns, based on their SMS text messages in a large, digital mental health service platform (ie, Talkspace). This platform uses natural language processing methods to analyze unstructured therapy transcript data, in parallel with brief clinical assessment methods for analyzing depression and anxiety symptoms. </sec> <sec> <title>RESULTS</title> Our results show a significant increase in the incidence of COVID-19–related intake anxiety symptoms (&lt;i&gt;P&lt;/i&gt;&amp;lt;.001), but no significant differences in the incidence of intake depression symptoms (&lt;i&gt;P&lt;/i&gt;=.79). During our transcript analyses, we identified terms that were related to 24 symptoms outside of those included in the diagnostic criteria for anxiety and depression. </sec> <sec> <title>CONCLUSIONS</title> Our findings for Talkspace suggest that people who seek treatment during the pandemic experience more severe intake anxiety than they did before the COVID-19 outbreak. It is important to monitor the symptoms that we identified in this study and the symptoms of anxiety and depression, to fully understand the effects of the COVID-19 pandemic on mental health. </sec>
BibTeX
@misc{hull2020analyzing,
  author = {Thomas D. Hull and Jacob Levine and Niels Bantilan and Angel N. Desai and Maimuna S. Majumder},
  title = {{Analyzing Digital Evidence From a Telemental Health Platform to Assess Complex Psychological Responses to the COVID-19 Pandemic: Content Analysis of Text Messages (Preprint)}},
  year = {2020},
  doi = {10.2196/preprints.26190},
}
Figure