WeChat assisted electronic symptom measurement for patients with adenomyosis

Wei Xu, Xin Zhang, Fan Xu, Yuan Yuan, Ying Tang, Qiuling ShiBMC Medical Informatics Decis. Mak 2024 · DOI

TL;DR

This paper introduces the AM-SAS, a novel electronic symptom assessment tool for adenomyosis patients, developed using WeChat data and validated through rigorous psychometric testing. It offers a more precise and timely way to monitor symptoms, potentially improving patient care and identifying disease recurrence.

What Problem Does This Paper Address?

Adenomyosis, a common gynecological condition, significantly impacts women's quality of life due to symptoms like heavy bleeding and pelvic pain. Effective management is crucial, especially for women of childbearing age who wish to preserve their uterus. However, current symptom assessment methods are often subjective, lack standardization, and don't fully capture the nuances of adenomyosis-specific symptoms. This leads to challenges in timely diagnosis, effective long-term management, and early detection of disease recurrence. Existing patient-reported outcome (PRO) measures, like the UFS-QOL, are not specific enough for adenomyosis. Furthermore, the reliance on traditional interviews or questionnaires can be time-consuming and may not reflect real-world symptom experiences as captured in informal communication channels.

Methods

The researchers developed the Adenomyosis Symptom Assessment Scale (AM-SAS) through a multi-stage process:

  • Concept Elicitation & Item Generation:
    • Qualitative Interviews: In-depth interviews with 29 adenomyosis patients were conducted to gather initial symptom descriptions.
  • Natural Language Processing (NLP) on WeChat Data: A massive dataset of 9,074 doctor-patient communication screenshots from WeChat was analyzed using NLP techniques. This approach aimed to capture real-world, unprompted symptom reporting. Keywords were extracted using TF-IDF.
  • Expert Panel Review: A panel of 86 gynecologists reviewed and rated the relevance of the generated symptom items.
  • Instrument Development & Refinement:
    • Drafting AM-SAS v1: Based on interview and WeChat data, an initial draft of 40 items was created.
  • Expert Reduction: The expert panel reduced the items to 18 (AM-SAS v2) based on relevance ratings. Further reduction to 16 items (AM-SAS v3) occurred after expert discussions on clinical management.
  • Cognitive Debriefing: 140 patients participated in cognitive debriefing to ensure items were understandable and relevant.
  • Final AM-SAS (10 items): Through further refinement, including combining similar symptoms (e.g., abdominal pain and bearing-down pain into abdominal cramping) and eliminating less burdensome symptoms (e.g., nausea/vomiting), a 10-item scale was finalized.
  • Psychometric Validation:
    • Classical Test Theory (CTT) & Item Response Theory (IRT): The AM-SAS was validated using both CTT (Cronbach's alpha for internal consistency, test-retest reliability) and IRT (Rasch analysis for unidimensionality and item fit).
  • Concurrent Validity: Correlated with the SF-36 to assess its relationship with general health status.
  • Known-Group Validity: Tested its ability to differentiate between groups with varying pain interference and anemia severity.
  • External Validation: The final 10-item AM-SAS was validated in a separate cohort to assess its performance in a real-world setting, including its ability to distinguish between patients with and without recurrence.

Datasets

  • WeChat Communication Screenshots: 9,074 screenshots from doctor-patient communication groups, containing 1,189,413 words, were used for NLP-based symptom extraction.
  • Patient Interviews: Data from 29 patients who underwent qualitative interviews.
  • Expert Panel: Data from 86 gynecologists in the first round and 46 in the second round for item relevance rating.
  • Cognitive Debriefing Participants: 140 patients.
  • Psychometric Validation Cohort (Cohort 3): 179 patients with adenomyosis.
  • External Validation Cohort: 130 post-treatment patients.

Key Insights and Ideas

  • Leveraging Social Media Data for Health: The study demonstrates the power of using informal communication channels like WeChat for capturing authentic, unprompted patient symptom data, complementing traditional research methods.
  • NLP for Clinical Insights: NLP proved effective in extracting relevant symptoms from noisy, unstructured text data, highlighting its potential in medical research.
  • Unidimensionality of AM-SAS: Rasch analysis confirmed that the 10-item AM-SAS effectively measures a single underlying construct: adenomyosis symptom severity.
  • Key Index Symptoms: Menorrhagia and abdominal cramping emerged as particularly severe and prevalent symptoms, and importantly, showed a strong correlation with disease recurrence, suggesting their utility as index symptoms for monitoring.
  • WeChat as a Viable PRO Platform: The study validates the use of a WeChat mini-program for delivering and collecting PRO data, offering a convenient and accessible platform for patients.

Key Results

  • AM-SAS Development: A 10-item AM-SAS was developed, demonstrating good internal consistency (Cronbach's alpha > 0.8) and test-retest reliability (ICC > 0.7 for most items).
  • Psychometric Properties: The AM-SAS showed good unidimensionality, item fit, and construct validity.
  • Concurrent Validity: Spearman correlations showed moderate correlations between AM-SAS scores and SF-36 subscales.
  • Known-Group Validity: The AM-SAS was sensitive to differences in symptom severity based on pain interference and anemia status. Patients with anemia reported significantly higher menorrhagia severity (mean difference -2.63, P<0.001).
  • External Validation: The AM-SAS successfully distinguished between patients with and without recurrence, with those experiencing recurrence showing higher symptom burden (P<0.001).
  • Symptom Prevalence: Abdominal cramping and menorrhagia were identified as the most severe symptoms, with over 60% of patients reporting moderate-to-severe abdominal cramping and over 50% reporting menorrhagia.

Limitations and Future Work

  • Limited Follow-up Time: The study had a relatively short accrual period (6 months), limiting long-term symptom burden analysis. The authors plan to extend monitoring to one year.
  • External Validation Cohort Characteristics: The external validation cohort consisted of post-surgery patients with well-controlled disease, which may not fully represent the broader adenomyosis patient population.
  • Index Symptom Validation: Further clinical studies are needed to validate the proposed index symptoms (menorrhagia and abdominal cramping) for screening and recurrence monitoring.
  • Discrimination from Other Conditions: Future research should explore the AM-SAS's ability to differentiate adenomyosis patients from those with similar conditions like endometriosis.
  • AI Integration: The authors suggest exploring artificial intelligence (AI) for more personalized and proactive follow-up, including predictive analytics for menstrual cycles.
  • NLP vs. Interviews: The paper raises the question of whether NLP-extracted symptoms can fully replace expert interviews and manual identification for PRO development in the future.

Summary updated

BibTeX
@article{xu2024wechat,
  author = {Wei Xu and Xin Zhang and Fan Xu and Yuan Yuan and Ying Tang and Qiuling Shi},
  title = {{WeChat assisted electronic symptom measurement for patients with adenomyosis}},
  journal = {BMC Medical Informatics Decis. Mak.},
  year = {2024},
  doi = {10.1186/s12911-024-02570-8},
}
Figure