KidsTube: Detection, Characterization and Analysis of Child Unsafe Content & Promoters on YouTube

Rishabh Kaushal, Srishty Saha, Payal Bajaj, Ponnurangam Kumaraguru

Abstract

YouTube is a primary destination for children's entertainment, yet its open-access model allows malicious actors to upload inappropriate content or post toxic comments near popular cartoons. This research introduces a comprehensive framework for detecting "child unsafe" content and the users who promote it. By leveraging supervised machine learning, deep learning, and network analysis, the authors achieve an 85.7% detection accuracy. The study further characterizes the behavior of unsafe content promoters, revealing their low-engagement profiles and the "unsafe loops" created by recommendation algorithms.

The Problem: Malicious Content in Safe Spaces

While YouTube hosts legitimate children's programming, it is also a "minefield" where malicious users "piggyback" on popular keywords like "Mickey Mouse" or "Tom and Jerry." This results in two primary threats: indecent videos disguised as cartoons and abusive comments on safe video pages. Current reactive moderation systems, like the YouTube Kids App, often fail to catch these issues before exposure occurs.

Examples of (a) safe content, (b) unsafe content, and (c) toxic comments on safe videos.

Methodology

The researchers utilized a multi-tiered approach to identify unsafe content:

  1. Supervised Classification: A Random Forest classifier was trained on 34 features, including video metadata (view counts, title similarity), user metrics (upload history, social network stats), and comment sentiment analysis.
  2. Deep Learning (CNN): A Convolutional Neural Network was employed to analyze video frames for inappropriate visual transitions, such as sudden nudity.
  3. Network Analysis: The Louvain Modularity algorithm was used to map relationships between videos and users, identifying how unsafe content clusters within the broader ecosystem.

Characterizing Content Promoters

The study found that unsafe content promoters are generally less popular and engage less with the platform than legitimate creators. However, they are not isolated; they form close-knit communities that sit in close proximity to safe content.

Distribution and degree of unsafe content promotion among uploaders.

Analysis of the "Extended Dataset" (over 155,000 videos) showed that while moving from safe to unsafe content is relatively rare, the reverse is also true. Once a user enters an "unsafe" cluster, the recommendation engine tends to keep them there—a phenomenon the authors call the "unsafe loop."

Key Results and Performance

  • High Detection Accuracy: The Random Forest classifier achieved a peak accuracy of 85.7%, with 88.8% precision.
  • Comment Toxicity: Over 1,700 unique unsafe commenters were identified, many of whom targeted safe uploader pages, effectively rendering safe videos inappropriate for children.
  • CNN vs. Metadata: While the CNN achieved 74% accuracy, it was found to be computationally expensive compared to metadata-based classification.

Conclusion

This research highlights the urgent need for proactive, automated moderation on video-sharing platforms. By identifying the specific behavioral patterns and network structures of unsafe content promoters, platform providers can build more robust systems to shield young audiences from inappropriate material and the algorithmic traps that perpetuate its visibility.

Summary updated

Abstract

YouTube draws large number of users who contribute actively by uploading videos or commenting on existing videos. However, being a crowd sourced and large content pushed onto it, there is limited control over the content. This makes malicious users push content (videos and comments) which is inappropriate (unsafe), particularly when such content is placed around cartoon videos which are typically watched by kids. In this paper, we focus on presence of unsafe content for children and users who promote it. For detection of child unsafe content and its promoters, we perform two approaches, one based on supervised classification which uses an extensive set of video-level, user-level and comment-level features and another based Convolutional Neural Network using video frames. Detection accuracy of 85.7% is achieved which can be leveraged to build a system to provide a safe YouTube experience for kids. Through detailed characterization studies, we are able to successfully conclude that unsafe content promoters are less popular and engage less as compared with other users. Finally, using a network of unsafe content promoters and other users based on their engagements (likes, subscription and playlist addition) and other factors, we find that unsafe content is present very close to safe content and unsafe content promoters form very close knit communities with other users, thereby further increasing the likelihood of a child getting getting exposed to unsafe content.
BibTeX
@inproceedings{kaushal2016kidstube,
  author = {Rishabh Kaushal and Srishty Saha and Payal Bajaj and Ponnurangam Kumaraguru},
  title = {{KidsTube: Detection, Characterization and Analysis of Child Unsafe Content \& Promoters on YouTube}},
  booktitle = {PST},
  year = {2016},
  url = {https://openreview.net/forum?id=BidIOQ7jX_},
}
Figure