Giving a voice to adults with COVID-19: An analysis of open-ended comments from smell longhaulers and non-longhaulers

Nick Simon Menger, Arnaud Tognetti, Michael C. Farruggia, Carla Mucignat‐Caretta, Surabhi Bhutani, Keiland W Cooper, Paloma Rohlfs Domínguez, Thomas Heinbockel, Vonnie D. C. Shields, Antonietta D’Errico, Veronica Pereda‐Loth, Denis Pierron, Sachiko Koyama, Ilja CroijmansmedRxiv 2023 · DOI

TL;DR

Researchers used natural language processing (NLP) and manual coding to analyze open-ended survey comments from over 1,500 COVID-19 survivors. The study reveals that "smell longhaulers"—those with persistent olfactory dysfunction—experience significantly higher rates of parosmia (distorted smell) and express much higher levels of emotional distress compared to those who recovered quickly.

What Problem Does This Paper Address?

While we know that COVID-19 causes smell loss, standard multiple-choice surveys often fail to capture the nuance of the patient experience. Many patients lack the clinical vocabulary to distinguish between anosmia (total loss), hyposmia (partial loss), and parosmia (distorted, often foul smells). For instance, a patient might report losing their sense of "taste" when they actually mean "flavor," which is primarily driven by smell.

By analyzing open-ended comments, the authors sought to "give a voice" to these patients. They aimed to bridge the gap between clinical definitions and lived experiences, specifically looking at how long-term smell dysfunction impacts emotional well-being and daily interactions with food and the environment.

Methods

The researchers employed a multi-pronged analytical approach:

  1. Manual Quantitative Content Analysis: Eight coders manually reviewed 2,543 comments to identify specific dysfunctions (anosmia, hyposmia, parosmia, phantosmia) and mentions of food/non-food items. They used Fleiss’s kappa to ensure inter-coder reliability.
  2. Aspect-Based Sentiment Analysis (ABSA): Using the LCF-ATEPC algorithm from the PyABSA framework, the team classified the sentiment (positive, negative, or neutral) associated with specific "aspects" or keywords within the comments.
  3. Linguistic Frequency Analysis: The team used log-likelihood statistics to identify which words were significantly "overused" by longhaulers versus non-longhaulers.
  4. Visualization: Word clouds were generated to provide a qualitative snapshot of the most salient odor objects for different groups.

Datasets

  • Primary Data: Longitudinal survey data from the Global Consortium for Chemosensory Research (GCCR), involving 1,560 participants who provided comments in English, Dutch, French, Italian, or Spanish.
  • ML Training Sets: The sentiment analysis models were pre-trained on standard benchmarks: "Restaurant16" (restaurant reviews) and "Laptop14" (laptop reviews) to validate the sentiment classification across different domains.

Key Insights and Ideas

  • The Parosmia-Longhauler Link: Parosmia (distorted smell) is a hallmark of the longhauler experience. While anosmia is common in the acute phase, longhaulers are nearly twice as likely to report that things smell "wrong" or "foul" rather than just "weak."
  • The Emotional Toll: Sentiment analysis confirmed that longhaulers don't just have a sensory problem; they have an emotional one. Their comments were significantly more negative, reflecting the "psychological distress" of living in a world that smells like chemicals or rotting meat.
  • Recovery vs. Distortion Keywords:
    • Non-longhaulers frequently mentioned "wine," "cheese," and "coffee" in the context of regaining these scents.
  • Longhaulers overused words like "weird," "chemical," "smoke," and "burning." Interestingly, "lemon" was a frequent trigger for longhaulers, suggesting certain molecular compounds (like citral) might be common triggers for parosmia.
  • The "Flavor" Confusion: The text analysis highlighted how often patients report a loss of taste when their comments clearly describe a loss of retronasal olfaction (the ability to smell food while eating).

Key Results

  • Prevalence: Longhaulers were significantly more likely to report parosmia (OR = 1.78) and hyposmia (OR = 1.74) in their open-ended text compared to non-longhaulers.
  • Sentiment: A Chi-square test showed a strong relationship between longhauler status and negative sentiment ($p < .001$). In longhaulers, the presence of parosmia specifically predicted a more negative emotional tone ($\beta = -1.13, p = .001$).
  • Validation: The researchers found a discrepancy between multiple-choice answers and open-ended text. Coders often identified symptoms in the text that participants didn't check off in the boxes, suggesting that open-ended comments are a more sensitive (though harder to analyze) tool for clinical assessment.

Limitations and Future Work

  • Selection Bias: The study relies on a self-selected group of survey respondents. People with more severe or frustrating symptoms are more likely to take the time to write detailed comments.
  • Sentiment Model Constraints: The ABSA models were trained on restaurant and laptop reviews. While they worked well, they aren't optimized for medical or sensory-loss contexts, which may have led to some misclassifications.
  • Future Directions: The authors suggest that future research should focus on the "molecular triggers" identified in these comments (like the pyrazines in coffee or disulfides in onions) to better understand the biological mechanism of parosmia. They also advocate for using these NLP techniques to help clinicians better "translate" patient descriptions into accurate diagnoses.

Summary updated

Abstract

Abstract Smell disorders are commonly reported with COVID-19 infection. Some patients show prolonged smell-related issues, even after the respiratory symptoms are resolved. To explore the concerns of patients, and to provide an overview for each specific smell disorder, we explored the longitudinal survey that was conducted by 1 , and contained self-reports on the changes of smell that participants experienced at two time points. People who still suffered from smell disorders at the second time point, hence named ‘longhaulers’, were compared to those who were not, hence named ‘non-longhaulers’. Specifically, three aims were pursued in this study. First, to classify smell disorders based on the participants’ self-reports. Second, to classify the sentiment of each self-report using a machine learning approach, and third, to find specific keywords that best describe the smell dysfunction in those self-reports. We found that the prevalence of parosmia and hyposmia was higher in longhaulers than in non-longhaulers. Furthermore, the results suggest that longhaulers stated self-reports with more negative sentiment than non-longhaulers. Finally, we found specific keywords that were more typical for either longhaulers compared to non-longhaulers. Taken together, our work shows consistent findings with previous studies, while at the same time, provides new insights for future studies investigating smell disorders.
BibTeX
@misc{menger2023giving,
  author = {Nick Simon Menger and Arnaud Tognetti and Michael C. Farruggia and Carla Mucignat‐Caretta and Surabhi Bhutani and Keiland W Cooper and Paloma Rohlfs Domínguez and Thomas Heinbockel and Vonnie D. C. Shields and Antonietta D’Errico and Veronica Pereda‐Loth and Denis Pierron and Sachiko Koyama and Ilja Croijmans},
  title = {{Giving a voice to adults with COVID-19: An analysis of open-ended comments from smell longhaulers and non-longhaulers}},
  year = {2023},
  doi = {10.1101/2023.03.07.23286910},
  howpublished = {medRxiv},
}
Figure