Unveiling Voices: Identification of Concerns in a Social Media Breast Cancer Cohort via Natural Language Processing

Swati Rajwal, Avinash Kumar Pandey, Zhishuo Han, Abeed SarkerCOLING 2024

TL;DR

This paper introduces a novel NLP framework to analyze breast cancer (BC) patient concerns expressed on social media, aiming to understand reasons for treatment discontinuation. By leveraging RoBERTa for cohort identification and BERTopic for topic modeling, the study uncovers a wide range of patient anxieties beyond typical treatment side effects, including those related to COVID-19, insurance, and socio-emotional distress.

What Problem Does This Paper Address?

Continuity of long-term cancer therapies is crucial for preventing recurrence, yet many patients discontinue treatment prematurely due to adverse patient-centered outcomes (PCOS). These outcomes encompass a broad spectrum of experiences, from treatment side effects to broader socio-economic concerns, which are often underrepresented in traditional electronic health records. The authors aim to bridge this gap by exploring social media, specifically Twitter (now X), to capture these nuanced patient concerns and identify potential drivers of treatment non-compliance.

Methods

The study employs a multi-stage NLP pipeline. First, a BC self-report identification system was developed using supervised machine learning models, with RoBERTa demonstrating superior performance for classifying tweets as originating from BC patients. This identified cohort was then analyzed for concerns and sentiments. BERTopic was utilized for topic modeling to identify key issues and concerns within the BC patient discussions. Sentiment analysis was performed using NLTK's Sentiment Analyzer, focusing on negative sentiment posts to pinpoint potential reasons for treatment discontinuation. A Word2Vec embedding model was also used to curate keywords for topic identification.

Datasets

The primary dataset used is a collection of approximately 1.5 million tweets from 583,962 unique users, previously curated by Al-Garadi et al. (2020). A manually annotated subset of this data (Train=3513, Dev=302, Test=1204) was used for evaluating various classification models. No new benchmark datasets were introduced.

Key Insights and Ideas

  • Social Media as a Rich Source for PCOS: The study confirms that social media platforms are valuable for capturing patient-centered outcomes that are often missed in clinical settings.
  • Beyond Physical Side Effects: Patient concerns extend far beyond the physical manifestations of treatment. The research highlights significant anxieties related to the COVID-19 pandemic's impact on healthcare access, insurance complexities, and the psychological toll of the disease.
  • The Interplay of Social Determinants and Health: Issues like racial disparities in mortality and the fear of sexual abuse intersect with BC experiences, underscoring the need for a holistic understanding of patient well-being.
  • RoBERTa's Dominance in Cohort Identification: The transformer-based RoBERTa model significantly outperformed traditional machine learning classifiers in accurately identifying BC patient self-reports from social media.

Key Results

  • Cohort Identification: RoBERTa achieved a micro-averaged F₁ score of 0.894 and a macro-averaged F₁ score of 0.853, identifying 154,571 self-reported BC posts from the ~1.5 million tweet dataset.
  • Sentiment Analysis: Sentiment analysis of self-reported BC posts revealed a strong skew towards negative sentiments, with most scores falling at or below 0.5, indicating prevalent challenges and adverse experiences.
  • Key Concerns Identified: Topic modeling revealed several prominent concerns:
  • Impact of COVID-19 on Screenings: Fear of seeking medical help leading to delayed diagnoses.
  • Mammogram Anxiety: Anxiety surrounding screening, exacerbated by the pandemic and fear of discovery.
  • Insurance Issues: Frustration over lack of coverage and financial burdens.
  • Dietary Habits: Confusion regarding soy consumption and its alleged effects.
  • Racial Disparity in Mortality: Concerns about the disproportionate impact of BC on African-American women.
  • Hair/Baldness: Psychological and social impact of chemotherapy-induced alopecia.
  • Smoking Restrictions: Difficulty adhering to cessation protocols.
  • Bowel/Colon Issues: Intolerable gastrointestinal side effects.
  • Sexual Abuse Concerns: Traumatic personal histories intersecting with treatment.
  • Mental/Emotional Health: Significant emotional and psychological toll.

Limitations and Future Work

The authors acknowledge that for comprehensive longitudinal sentiment analysis, a sufficient number of posts per user is ideal, but their dataset had scarcity in this regard, leading to a consolidated sentiment analysis. Overlap with discussions of other health issues (like COVID-19) also affected the precision of isolating BC-specific conversations.

Future work could involve:

  • Expanding experiments to other social media platforms like Reddit and Facebook.
  • Analyzing data from different time periods to capture temporal trends.
  • Collecting long-term data to study temporal trends in patient concerns.
  • Further improving cohort creation by annotating more data to enhance accuracy.

Summary updated

BibTeX
@inproceedings{rajwal2024unveiling,
  author = {Swati Rajwal and Avinash Kumar Pandey and Zhishuo Han and Abeed Sarker},
  title = {{Unveiling Voices: Identification of Concerns in a Social Media Breast Cancer Cohort via Natural Language Processing}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
}
Figure