YouTube for Patient Education: A Deep Learning Approach for Understanding Medical Knowledge from User-Generated Videos

Xiao Liu, Bin Zhang, Anjana Susarla, Rema PadmanarXiv 2018 · arXiv

Introduction

YouTube has become a primary source of health information for the general public, yet the quality of its content remains highly variable. With only 12% of the US population possessing proficient health literacy, many users are vulnerable to misleading or low-quality medical advice found in user-generated videos. While expert-led clinical tutorials exist, they are often buried under a mountain of anecdotal "homemade" content.

Because the volume of uploads makes manual expert review impossible, researchers have developed an automated, multimodal deep learning framework to evaluate the medical knowledge level of these videos. This system aims to bridge the health literacy gap by algorithmically distinguishing between high-quality educational content and low-knowledge videos with high accuracy.

Methodology

The proposed framework utilizes an interdisciplinary approach that extracts knowledge from both the audio-visual and textual components of a video. The process is divided into three main stages: text-based extraction, image-based extraction, and final classification.

Multimodal framework for medical knowledge assessment in YouTube videos.

Text-based Medical Knowledge Assessment

To process the linguistic content, the system uses Natural Language Processing (NLP) techniques:

  • Word Embeddings: Word2vec (Skip-gram) was trained on nearly 20,000 video descriptions to create 50-dimensional semantic vectors, capturing the context of medical terminology.
  • BLSTM-RNN: A Bidirectional Long Short-Term Memory (BLSTM) model performs Named Entity Recognition (NER). By analyzing context both before and after a word, it identifies specific medical terms (e.g., "pathogenesis" or "insulin") more effectively than traditional dictionary-matching tools.

Image-based Medical Knowledge Extraction

To understand the visual context, the system employs computer vision:

  • CNN Architecture: A pre-trained Inception Convolutional Neural Network (CNN) performs object detection on video frames sampled every two seconds.
  • Feature Engineering: The model identifies the top five object categories per frame, which are then filtered for medical relevance (e.g., recognizing medical devices or clinical settings) to serve as features for classification.

Data Collection and Annotation

The study focused on Diabetes, initially scraping 19,873 unique videos. A subset of 600 videos with closed captions was selected for detailed analysis. Two graduate researchers annotated medical terms and labeled knowledge levels, with a medical doctor serving as the domain expert to consolidate annotations. The inter-rater reliability was high, reaching 0.87 for term annotation and 0.92 for knowledge classification. Additionally, over 200,000 individual frames were extracted for the image-based analysis.

Key Results and Insights

The framework demonstrated significant improvements over existing industry standards. The BLSTM method for medical term extraction achieved a 92.9% F-measure, vastly outperforming MetaMap (28.9%).

For the final task of categorizing videos into "High" versus "Low" medical knowledge, the integrated Logistic Regression classifier achieved an overall accuracy of 85%.

Classification performance for high and low medical knowledge videos.

Key insights from the study include:

  • Context is King: Deep learning models like BLSTM are far superior to simple lexicons for user-generated content, which often uses informal or non-standard language.
  • Multimodal Synergy: Combining what is said (text) with what is shown (images) provides a more holistic understanding of a video's educational value.

Limitations and Future Work

While promising, the current model is a binary classifier (High/Low). Future iterations aim to provide more granular quality scores to match varying levels of patient health literacy. Additionally, the researchers plan to fine-tune the vision models on specialized medical datasets to improve the recognition of clinical equipment and expand the scope beyond diabetes to other chronic conditions. The ultimate goal is to integrate these "quality scores" into search and recommendation algorithms to prioritize reputable health information for all users.

Summary updated

Abstract

YouTube presents an unprecedented opportunity to explore how machine learning methods can improve healthcare information dissemination. We propose an interdisciplinary lens that synthesizes machine learning methods with healthcare informatics themes to address the critical issue of developing a scalable algorithmic solution to evaluate videos from a health literacy and patient education perspective. We develop a deep learning method to understand the level of medical knowledge encoded in YouTube videos. Preliminary results suggest that we can extract medical knowledge from YouTube videos and classify videos according to the embedded knowledge with satisfying performance. Deep learning methods show great promise in knowledge extraction, natural language understanding, and image classification, especially in an era of patient-centric care and precision medicine.
BibTeX
@misc{liu2018youtube,
  author = {Xiao Liu and Bin Zhang and Anjana Susarla and Rema Padman},
  title = {{YouTube for Patient Education: A Deep Learning Approach for Understanding Medical Knowledge from User-Generated Videos}},
  year = {2018},
  eprint = {1807.03179},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1807.03179},
}
Figure