Finding influential users of an online health community: a new metric based on sentiment influence

Kang Zhao, Greta E. Greer, Baojun Qiu, Prasenjit Mitra, Kenneth Portier, John YenarXiv 2012 · arXiv

Abstract

Identifying influential users (IUs) in online health communities (OHCs) is critical for maintaining community resilience and support. Traditional metrics focusing on activity volume or network centrality often fail to capture the emotional support core to these communities. This paper introduces a novel metric, Influential Responding Replies (IRR), which identifies leaders based on their ability to shift the sentiment of peers. Using a decade of data from the Cancer Survivors Network, we demonstrate that IRR significantly outperforms traditional social network analysis metrics, achieving high precision and recall in identifying community-validated leaders.

Introduction

Online health communities (OHCs) serve as vital psychosocial lifelines for patients and caregivers. These platforms rely on "influential users" (IUs)—mentors who provide emotional and informational support. However, because these leaders are often patients themselves, their sudden absence due to health declines can destabilize the community. Automatically identifying these individuals is a challenge; traditional methods often prioritize "loudness" (post frequency) or "connectivity" (follower count), which may not reflect actual supportive influence. This research shifts the focus to sentiment influence, measuring how a user’s response effectively changes a peer's emotional state.

To validate this approach, the study utilized a massive longitudinal dataset from the Cancer Survivors Network (CSN), spanning 10 years and comprising over 468,000 posts. Researchers worked with community managers to establish a "gold standard" list of 126 known influential users for performance benchmarking.

Methods

The researchers developed the Influential Responding Replies (IRR) metric, which operates through a multi-stage pipeline.

Sentiment Classification

First, a machine-learning classifier (AdaBoost with regression trees) was trained to label posts as having positive or negative sentiment. The model achieved approximately 79% accuracy, effectively handling the nuanced language of health forums.

Sentiment classification and influence modeling pipeline.

Defining and Measuring Influence

The study analyzed threaded discussions, focusing on the interaction between an "originator" (the person starting the thread) and respondents. Influence was defined by the originator's sentiment shift following a reply. A reply is categorized as an IRR if it occurs before the originator's first self-reply and if the originator’s subsequent sentiment aligns with the respondent's sentiment (e.g., a positive reply leading to a more positive self-reply).

Key Insights: The "Golden Window"

A significant discovery was the identification of a "Golden Window" for influence. Data analysis revealed that the most substantial sentiment changes occur between the initial post and the very first self-reply.

Sentiment indicator (Pr) as a function of the post sequence from the thread originator.

As shown in the figure above, the sentiment indicator ($P_r$) tends to stabilize after the initial interactions. This suggests that first responders have the highest impact on a user's emotional trajectory within a thread.

Results

The IRR metric significantly outperformed traditional social network analysis (SNA) metrics when compared against the community-validated list of leaders:

  • Superior Precision: In the Top-50 ranking, the IRR metric achieved a precision of 0.880, outperforming metrics like PageRank or Out-degree, which often flag high-volume "chatterboxes" rather than true influencers.
  • High Recall: When integrated into an ensemble classifier (Random Forest), the model achieved a perfect 1.000 recall for the Top-150 users, successfully identifying every community-nominated leader.
  • Sentiment Alignment: Unlike simple positivity metrics, IRR accounts for "healthy negativity"—such as shared grief—by measuring sentiment alignment rather than just a shift toward positive language.

Discussion and Future Work

While the IRR metric provides a robust tool for identifying emotional leaders, limitations remain. The model cannot account for offline influences (e.g., medical visits) and does not yet distinguish between "healthy" and "unhealthy" negative influence (e.g., arguments). Future research will aim to incorporate "informational influence" to identify users who provide critical medical and navigational advice alongside emotional support.

Reference: Zhao, K., Greer, G., Qiu, B., Mitra, P., Portier, K., & Yen, J. Finding influential users of an online health community: a new metric based on sentiment influence. Working Paper.

Summary updated

Abstract

What characterizes influential users in online health communities (OHCs)? We hypothesize that (1) the emotional support received by OHC members can be assessed from their sentiment ex-pressed in online interactions, and (2) such assessments can help to identify influential OHC members. Through text mining and sentiment analysis of users' online interactions, we propose a novel metric that directly measures a user's ability to affect the sentiment of others. Using dataset from an OHC, we demonstrate that this metric is highly effective in identifying influential users. In addition, combining the metric with other traditional measures further improves the identification of influential users. This study can facilitate online community management and advance our understanding of social influence in OHCs.
BibTeX
@misc{zhao2012finding,
  author = {Kang Zhao and Greta E. Greer and Baojun Qiu and Prasenjit Mitra and Kenneth Portier and John Yen},
  title = {{Finding influential users of an online health community: a new metric based on sentiment influence}},
  year = {2012},
  eprint = {1211.6086},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1211.6086},
}
Figure