Exploring Semantic Topics in Dementia Caregiver Tweets

Yanbo Feng, Bojian Hou, Ari Z Klein, Karen O’Connor, Jiong Chen, Andrés Mondragón, Shuhua Yang, Graciela Gonzalez‐Hernandez, Li ShenAlzheimer 2024 · DOI

TL;DR

This paper introduces a novel topic modeling approach, Log-BDC DMM, to analyze semantic themes in dementia caregiver tweets. By combining advanced weighting strategies with a Dirichlet Multinomial Mixture model, they achieved superior performance in extracting meaningful topics from noisy social media data, offering valuable insights into caregiver concerns.

What Problem Does This Paper Address?

Caring for individuals with dementia is a significant and growing challenge, and understanding the needs and concerns of family caregivers is crucial for providing effective support. Social media platforms like Twitter offer a rich, albeit noisy, source of information about these caregivers' experiences. However, extracting coherent and meaningful insights from short, informal, and often ambiguous tweets is difficult. Traditional topic modeling techniques can struggle with the inherent noise and brevity of such data. This paper aims to address this by developing and evaluating a more robust topic modeling method capable of uncovering the semantic themes that matter most to dementia caregivers on Twitter.

Methods

The authors employed a multi-stage approach. First, they performed extensive quality control on the Twitter data, including spam removal, lemmatization, stopword removal, compound word segmentation, and spell checking (as illustrated in Figure 1). This preprocessing step is critical for cleaning the noisy tweet data.

The core of their methodology lies in applying topic modeling. They specifically focused on the Dirichlet Multinomial Mixture (DMM) model and introduced two novel word-frequency weighting strategies:

  • Log weighting: This strategy down-weights very frequent words, which are often less informative.
  • Balanced Distributional Concentration (BDC) weighting: This method aims to identify words that are both frequent within a topic and distinctive compared to other topics.

They combined these weighting strategies with the DMM model to create four variants (Log-DMM, BDC-DMM, Log-BDC-DMM, and a baseline DMM). These were compared against traditional LDA (Latent Dirichlet Allocation) variants and BERTopic variants (which leverage pre-trained language models). Model performance was evaluated using the C_V coherence score, a metric that assesses the interpretability and consistency of topics. Finally, GPT-4 was used to summarize the word bags from the best-performing model to generate human-readable semantic themes.

Datasets

The study utilized a dataset of 224,862 dementia-caregiver-related tweets. The data collection methodology is referenced as being available at https://aging.jmir.org/2022/3/e39547. No new benchmarks were introduced; rather, existing topic modeling approaches were used for comparison against their proposed method.

Key Insights and Ideas

  • The power of combined weighting: The Log and BDC weighting strategies, when applied together to the DMM model (Log-BDC DMM), significantly improved topic coherence and interpretability compared to using either strategy alone or standard DMM. This highlights the benefit of a nuanced approach to word importance in noisy text.
  • DMM's resilience to noise: The DMM model, especially with the proposed weighting, proved more effective than LDA and BERTopic variants in extracting meaningful themes from short and noisy tweets. This suggests that DMM's underlying assumptions or structure are better suited for this type of data.
  • GPT-4 as an interpretative aid: Leveraging GPT-4 for summarizing word bags from topics was highly effective in generating interpretable semantic themes, demonstrating the potential of large language models in bridging the gap between raw topic model outputs and human understanding.
  • Caregiver concerns are multifaceted: The identified semantic themes (Figure 3) reveal a broad spectrum of caregiver experiences, encompassing emotional struggles, daily activities, healthcare challenges, financial management, and the impact of external factors like COVID-19.

Key Results

  • The Log-BDC DMM model achieved the highest average C_V coherence score of 0.5648 across various topic numbers, outperforming all traditional LDA and BERTopic variants.
  • When the number of topics was set to 10, the Log-BDC DMM achieved a coherence score of 0.6054, which was the highest among all tested models and topic configurations.
  • Using GPT-4 to interpret the top 10 topics from the Log-BDC DMM model yielded 10 distinct and interpretable semantic themes related to dementia caregiving.

Limitations and Future Work

  • Initial Study: The authors acknowledge this is an initial study, suggesting that further exploration and validation are needed.
  • Model Interpretability: While the Log-BDC DMM model demonstrated high interpretability, the paper notes that maintaining high model interpretability alongside performance is a key aspect of their contribution.
  • Future Directions: The paper suggests that this approach has promising power for mining meaningful semantic topics from short and noisy tweets and can provide better care to family caregivers. Future work could involve applying this methodology to larger datasets, exploring different social media platforms, or investigating specific sub-groups of caregivers.

Summary updated

Abstract

Abstract Background Caring for family caregivers of dementia patients has grown to an important topic. Social media platforms, like Twitter, provide great resources for studying the needs of caregivers. It would be beneficial to understand the caregivers’ interested or concerned topics from their tweets. Meanwhile, topic modeling has become an efficient NLP tool for analyzing semantic themes within texts. Thus, in this study, we perform a novel topic model leveraging weighting strategies to analyze dementia‐caregiver‐related twitter data. Method The information about the twitter data collection is available at: https://aging.jmir.org/2022/3/e39547 . In our analysis, we first performed quality control, as shown in Figure 1. Then, we deployed two word‐frequency weighting strategies, Log weighting and Balanced Distributional Concentration (BDC) weighting , on the Dirichlet Multinomial Mixture (DMM) model to obtain 4 DMM variants and used them to mine semantic topics within the tweets. We compared the performance of DMM variants, LDA variants, and BERTopic variants. Topic numbers of 5, 10, 15, 20 were picked to achieve comprehensive comparison. The coherence score C_V was applied to evaluate the model performance. We then leveraged the GPT‐4 to summarize the bag of words in each topic from the best model to generate interpretable themes. Result 224,862 tweets passed the quality control and were analyzed in our study. As shown in Figure 2, the performance evaluation demonstrated that the Log‐BDC DMM has the best general performance among traditional and BERT‐based models, with an averaged coherence score of 0.5648. The bags of words from the Log‐BDC DMM with 10 topics were interpreted using GPT‐4 due to the highest coherence score of 0.6054. The interpretation yielded 10 semantic themes, as shown in Figure 3. Conclusion Our investigation of the twitter data shows that the DMM model with Log and BDC strategies has promising power for mining meaningful semantic topics from short and noisy tweets. The model outperforms LDA models and the BERTopic models while maintaining high model interpretability. This initial study on twitter data provides interesting findings that can provide better care to family caregivers and demonstrates the promise and interpretability of traditional topic modeling methods.
BibTeX
@inproceedings{feng2024exploring,
  author = {Yanbo Feng and Bojian Hou and Ari Z Klein and Karen O’Connor and Jiong Chen and Andrés Mondragón and Shuhua Yang and Graciela Gonzalez‐Hernandez and Li Shen},
  title = {{Exploring Semantic Topics in Dementia Caregiver Tweets}},
  booktitle = {Alzheimer},
  year = {2024},
  doi = {10.1002/alz.093035},
}
Figure