Abstract
YouTube is a primary destination for children's entertainment, yet its open-access model allows malicious actors to upload inappropriate content or post toxic comments near popular cartoons. This research introduces a comprehensive framework for detecting "child unsafe" content and the users who promote it. By leveraging supervised machine learning, deep learning, and network analysis, the authors achieve an 85.7% detection accuracy. The study further characterizes the behavior of unsafe content promoters, revealing their low-engagement profiles and the "unsafe loops" created by recommendation algorithms.
The Problem: Malicious Content in Safe Spaces
While YouTube hosts legitimate children's programming, it is also a "minefield" where malicious users "piggyback" on popular keywords like "Mickey Mouse" or "Tom and Jerry." This results in two primary threats: indecent videos disguised as cartoons and abusive comments on safe video pages. Current reactive moderation systems, like the YouTube Kids App, often fail to catch these issues before exposure occurs.
Methodology
The researchers utilized a multi-tiered approach to identify unsafe content:
- Supervised Classification: A Random Forest classifier was trained on 34 features, including video metadata (view counts, title similarity), user metrics (upload history, social network stats), and comment sentiment analysis.
- Deep Learning (CNN): A Convolutional Neural Network was employed to analyze video frames for inappropriate visual transitions, such as sudden nudity.
- Network Analysis: The Louvain Modularity algorithm was used to map relationships between videos and users, identifying how unsafe content clusters within the broader ecosystem.
Characterizing Content Promoters
The study found that unsafe content promoters are generally less popular and engage less with the platform than legitimate creators. However, they are not isolated; they form close-knit communities that sit in close proximity to safe content.
Analysis of the "Extended Dataset" (over 155,000 videos) showed that while moving from safe to unsafe content is relatively rare, the reverse is also true. Once a user enters an "unsafe" cluster, the recommendation engine tends to keep them there—a phenomenon the authors call the "unsafe loop."
Key Results and Performance
- High Detection Accuracy: The Random Forest classifier achieved a peak accuracy of 85.7%, with 88.8% precision.
- Comment Toxicity: Over 1,700 unique unsafe commenters were identified, many of whom targeted safe uploader pages, effectively rendering safe videos inappropriate for children.
- CNN vs. Metadata: While the CNN achieved 74% accuracy, it was found to be computationally expensive compared to metadata-based classification.
Conclusion
This research highlights the urgent need for proactive, automated moderation on video-sharing platforms. By identifying the specific behavioral patterns and network structures of unsafe content promoters, platform providers can build more robust systems to shield young audiences from inappropriate material and the algorithmic traps that perpetuate its visibility.
Summary updated