On Curating Responsible and Representative Healthcare Video Recommendations for Patient Education and Health Literacy: An Augmented Intelligence Approach

Krishna Pothugunta, Xiao Liu, Anjana Susarla, Rema PadmanarXiv 2022 · DOI · arXiv

Abstract

As digital platforms like YouTube become primary sources for health information, the gap between medical accuracy and patient understandability grows. This paper proposes an "augmented intelligence" framework designed to curate high-quality healthcare videos, specifically focusing on Type 2 Diabetes. By combining expert medical labeling with semi-supervised co-training, the researchers identify videos that are both medically sound and accessible to patients with varying health literacy levels. Furthermore, the framework audits the corpus for demographic biases in age and gender, revealing significant disparities in representation.

Introduction

While approximately one-third of US adults utilize the internet to diagnose health concerns, only 12% possess proficient health literacy. YouTube's recommendation algorithms frequently prioritize engagement metrics over medical accuracy or "understandability," creating a health literacy divide that disproportionately impacts disadvantaged populations. Addressing this requires a scalable solution that can vet millions of videos while ensuring recommendations are representative and unbiased.

Methodology

The proposed framework utilizes a human-in-the-loop, augmented intelligence approach to synthesize multiple machine learning techniques for video curation and bias auditing.

Overview of the augmented intelligence framework for healthcare video collection and analysis.

The process involves:

  • Data Collection: A corpus of 9,873 videos related to Type 2 Diabetes was gathered via the YouTube Data API using 200 health-related keywords.
  • Semi-Supervised Co-training: To mitigate the need for massive labeled datasets, a co-training model was employed. Two classifiers—one focusing on video metadata (logistic regression) and another on video content (random forest)—iteratively labeled the unlabeled pool based on high-confidence predictions.
  • Natural Language Processing (NLP): A Bidirectional Long Short-Term Memory (BiLSTM) model extracted medical terms from descriptions, mapped against the Unified Medical Language System (UMLS).
  • Computer Vision for Bias Auditing: FaceLib and pre-trained Caffe models were used to detect faces and estimate the age and gender of presenters to evaluate demographic representation.

Results and Performance

The co-training approach significantly outperformed individual baseline models. The framework achieved an overall accuracy of 87.5% in classifying medical information quality.

ROC curves comparing the performance of the co-training approach against standard machine learning models.

Human validation confirmed the model's reliability, with a Kappa score of 0.9 for medical term annotation and 0.87 for video understandability. Statistical analysis using Generalized Linear Models (GLM) indicated that "understandability" was the strongest predictor of video views (GLM = 0.731), suggesting a public preference for accessible content.

Demographic Insights and Bias

The audit revealed a pronounced gender gap within the diabetes video corpus. Most content categorized as having "high understandability" and "high medical information" featured male presenters. While face visibility itself did not significantly impact view counts, the gender of the presenter did (GLM = 0.635), with male presenters garnering more views on average. This suggests systemic biases in how health content is recommended or consumed.

Conclusion and Future Work

This research provides a blueprint for auditing "black box" recommendation algorithms for systemic bias. However, limitations remain, including potential algorithmic bias in the face detection tools themselves and a narrow focus on a single chronic condition. Future work will expand this framework to a broader range of health topics and conduct direct comparisons between "responsible" recommendations and standard platform baselines to further promote health equity.

Summary updated

Abstract

Studies suggest that one in three US adults use the Internet to diagnose or learn about a health concern. However, such access to health information online could exacerbate the disparities in health information availability and use. Health information seeking behavior (HISB) refers to the ways in which individuals seek information about their health, risks, illnesses, and health-protective behaviors. For patients engaging in searches for health information on digital media platforms, health literacy divides can be exacerbated both by their own lack of knowledge and by algorithmic recommendations, with results that disproportionately impact disadvantaged populations, minorities, and low health literacy users. This study reports on an exploratory investigation of the above challenges by examining whether responsible and representative recommendations can be generated using advanced analytic methods applied to a large corpus of videos and their metadata on a chronic condition (diabetes) from the YouTube social media platform. The paper focusses on biases associated with demographic characters of actors using videos on diabetes that were retrieved and curated for multiple criteria such as encoded medical content and their understandability to address patient education and population health literacy needs. This approach offers an immense opportunity for innovation in human-in-the-loop, augmented-intelligence, bias-aware and responsible algorithmic recommendations by combining the perspectives of health professionals and patients into a scalable and generalizable machine learning framework for patient empowerment and improved health outcomes.
BibTeX
@misc{pothugunta2022curating,
  author = {Krishna Pothugunta and Xiao Liu and Anjana Susarla and Rema Padman},
  title = {{On Curating Responsible and Representative Healthcare Video Recommendations for Patient Education and Health Literacy: An Augmented Intelligence Approach}},
  year = {2022},
  eprint = {2207.07915},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2207.07915},
}
Figure