Abstract
Identifying influential users (IUs) in online health communities (OHCs) is critical for maintaining community resilience and support. Traditional metrics focusing on activity volume or network centrality often fail to capture the emotional support core to these communities. This paper introduces a novel metric, Influential Responding Replies (IRR), which identifies leaders based on their ability to shift the sentiment of peers. Using a decade of data from the Cancer Survivors Network, we demonstrate that IRR significantly outperforms traditional social network analysis metrics, achieving high precision and recall in identifying community-validated leaders.
Introduction
Online health communities (OHCs) serve as vital psychosocial lifelines for patients and caregivers. These platforms rely on "influential users" (IUs)—mentors who provide emotional and informational support. However, because these leaders are often patients themselves, their sudden absence due to health declines can destabilize the community. Automatically identifying these individuals is a challenge; traditional methods often prioritize "loudness" (post frequency) or "connectivity" (follower count), which may not reflect actual supportive influence. This research shifts the focus to sentiment influence, measuring how a user’s response effectively changes a peer's emotional state.
To validate this approach, the study utilized a massive longitudinal dataset from the Cancer Survivors Network (CSN), spanning 10 years and comprising over 468,000 posts. Researchers worked with community managers to establish a "gold standard" list of 126 known influential users for performance benchmarking.
Methods
The researchers developed the Influential Responding Replies (IRR) metric, which operates through a multi-stage pipeline.
Sentiment Classification
First, a machine-learning classifier (AdaBoost with regression trees) was trained to label posts as having positive or negative sentiment. The model achieved approximately 79% accuracy, effectively handling the nuanced language of health forums.
Defining and Measuring Influence
The study analyzed threaded discussions, focusing on the interaction between an "originator" (the person starting the thread) and respondents. Influence was defined by the originator's sentiment shift following a reply. A reply is categorized as an IRR if it occurs before the originator's first self-reply and if the originator’s subsequent sentiment aligns with the respondent's sentiment (e.g., a positive reply leading to a more positive self-reply).
Key Insights: The "Golden Window"
A significant discovery was the identification of a "Golden Window" for influence. Data analysis revealed that the most substantial sentiment changes occur between the initial post and the very first self-reply.
As shown in the figure above, the sentiment indicator ($P_r$) tends to stabilize after the initial interactions. This suggests that first responders have the highest impact on a user's emotional trajectory within a thread.
Results
The IRR metric significantly outperformed traditional social network analysis (SNA) metrics when compared against the community-validated list of leaders:
- Superior Precision: In the Top-50 ranking, the IRR metric achieved a precision of 0.880, outperforming metrics like PageRank or Out-degree, which often flag high-volume "chatterboxes" rather than true influencers.
- High Recall: When integrated into an ensemble classifier (Random Forest), the model achieved a perfect 1.000 recall for the Top-150 users, successfully identifying every community-nominated leader.
- Sentiment Alignment: Unlike simple positivity metrics, IRR accounts for "healthy negativity"—such as shared grief—by measuring sentiment alignment rather than just a shift toward positive language.
Discussion and Future Work
While the IRR metric provides a robust tool for identifying emotional leaders, limitations remain. The model cannot account for offline influences (e.g., medical visits) and does not yet distinguish between "healthy" and "unhealthy" negative influence (e.g., arguments). Future research will aim to incorporate "informational influence" to identify users who provide critical medical and navigational advice alongside emotional support.
Reference: Zhao, K., Greer, G., Qiu, B., Mitra, P., Portier, K., & Yen, J. Finding influential users of an online health community: a new metric based on sentiment influence. Working Paper.
Summary updated