YouTubers Not madeForKids: Detecting Channels Sharing Inappropriate Videos Targeting Children

Myrsini Gkolemi, Panagiotis Papadopoulos, Evangelos P. Markatos, Nicolas Kourtellis

TL;DR

This paper investigates YouTube channels that share inappropriate videos targeting children, revealing a significant gap between YouTube's flagging system and the actual availability of harmful content. The authors develop a machine learning classifier that can predict potentially problematic channels at creation time, offering a proactive approach to content moderation.

What Problem Does This Paper Address?

YouTube has become a primary source of entertainment and information for children, but it also hosts a concerning amount of content that is inappropriate, disturbing, violent, or sexual. While YouTube has implemented various measures to combat this, including a "madeForKids" flag, the authors highlight a critical issue: a large percentage of videos and channels previously identified as inappropriate remain accessible on the platform. This research aims to bridge the gap between YouTube's automated systems and human judgment, and to proactively identify channels that are likely to host such content, even before they upload it.

Methods

The researchers employed a multi-stage methodology. First, they leveraged a pre-existing dataset of videos labeled as "disturbing" or "suitable" for children. They then crawled YouTube to determine the current availability of these videos and their associated channels. A key innovation was the analysis of a wide array of channel features, including metadata from the YouTube API (e.g., channel creation date, country, keywords, topics, view counts, subscriber counts) and data scraped from channel "About" and "Community" tabs (e.g., description sentiment, emotions, social media links, emoji usage). Finally, they used these features to train machine learning classifiers, including Random Forest and meta-classifiers, to predict whether a channel is likely to host inappropriate content. They also specifically explored the effectiveness of using only features available at channel creation time for early detection.

Datasets

The primary dataset used was derived from a previous study [27] that identified approximately 4797 YouTube videos and their associated labels (disturbing, suitable, restricted, irrelevant). For this study, the researchers focused on 2442 videos labeled as "disturbing" or "suitable," and the channels that hosted them. They then performed their own crawls in mid-2021 to assess the current status of these videos and channels. No new benchmark datasets were introduced, but the study provides a valuable update on the persistence of inappropriate content over time.

Key Insights and Ideas

  • Persistence of Harmful Content: A staggering ~60% of videos previously identified as disturbing in 2019 were still available on YouTube in mid-2021. Furthermore, nearly 71.5% of channels that uploaded such content had not been suspended. This highlights a significant challenge in content moderation.
  • "MadeForKids" Flag Inconsistency: While the "madeForKids" flag is intended to help creators and YouTube identify child-directed content, the study found that channels posting disturbing content are significantly less likely to be marked as "madeForKids" (3%) compared to suitable channels (25%). This suggests creators of inappropriate content may be deliberately avoiding this flag to evade scrutiny.
  • Predictive Channel Features: The research identified several channel characteristics that strongly correlate with the likelihood of hosting inappropriate content. These include a higher prevalence of gaming-related keywords and topics, a preference for Twitter over Facebook for social media links, and a tendency to use negative emotions (anger, disgust, sadness) in their descriptions.
  • Early Detection is Possible: Crucially, the study demonstrated that a machine learning classifier can effectively predict potentially disturbing channels using only features available at the time of channel creation, even before any videos are uploaded or subscribers are gained. This opens the door for proactive intervention.

Key Results

  • Content Availability: ~60% of disturbing videos and ~71.5% of disturbing channels from a 2019 study were still accessible in mid-2021.
  • "MadeForKids" Flag Usage: Only 3% of disturbing channels were marked as "madeForKids," compared to 25% of suitable channels.
  • Classifier Performance: Machine learning classifiers achieved high performance in detecting channels likely to share disturbing content.
  • A Random Forest classifier using all features achieved an AUC of up to 0.873.
  • A classifier using only channel creation-time features achieved an AUC of up to 0.869.

Limitations and Future Work

The authors acknowledge several limitations. The dataset is limited to channels that had previously uploaded videos identified as disturbing, introducing a selection bias. The research also grapples with the subjective nature of "disturbing" content, noting that some videos might be dark humor rather than outright harmful. Future work could involve developing more nuanced categorization of "disturbing" content and exploring better monitoring strategies beyond simple removal. The authors propose a multi-step process for YouTube to combine their findings on channel features and ML classifiers for more effective content moderation, including human review of high-ranked suspicious channels.

Summary updated

Abstract

In the last years, hundreds of new Youtube channels have been creating and sharing videos targeting children, with themes related to animation, superhero movies, comics, etc. Unfortunately, many of these videos are inappropriate for consumption by their target audience, due to disturbing, violent, or sexual scenes. In this paper, we study YouTube channels found to post suitable or disturbing videos targeting kids in the past. We identify a clear discrepancy between what YouTube assumes and flags as inappropriate content and channel, vs. what is found to be disturbing content and still available on the platform, targeting kids. In particular, we find that almost 60\% of videos that were manually annotated and classified as disturbing by an older study in 2019 (a collection bootstrapped with Elsa and other keywords related to children videos), are still available on YouTube in mid 2021. In the meantime, 44% of channels that uploaded such disturbing videos, have yet to be suspended and their videos to be removed. For the first time in literature, we also study the "madeForKids" flag, a new feature that YouTube introduced in the end of 2019, and compare its application to the channels that shared disturbing videos, as flagged from the previous study. Apparently, these channels are less likely to be set as "madeForKids" than those sharing suitable content. In addition, channels posting disturbing videos utilize their channel features such as keywords, description, topics, posts, etc., to appeal to kids (e.g., using game-related keywords). Finally, we use a collection of such channel and content features to train ML classifiers able to detect, at channel creation time, when a channel will be related to disturbing content uploads. These classifiers can help YouTube moderators reduce such incidences, pointing to potentially suspicious accounts without analyzing actual videos.
BibTeX
@inproceedings{gkolemi2022youtubers,
  author = {Myrsini Gkolemi and Panagiotis Papadopoulos and Evangelos P. Markatos and Nicolas Kourtellis},
  title = {{YouTubers Not madeForKids: Detecting Channels Sharing Inappropriate Videos Targeting Children}},
  booktitle = {WebSci},
  year = {2022},
  url = {https://openreview.net/forum?id=TWlUg2yeeH},
}
Figure