White Paper: Challenges and Considerations for the Creation of a Large Labelled Repository of Online Videos with Questionable Content

Thamar Solorio, Mahsa Shafaei, Christos Smailis, Mona T. Diab, Theodore Giannakopoulos, Heng Ji, Yang Liu, Rada Mihalcea, Smaranda Muresan, Ioannis A. KakadiarisarXiv 2021 · arXiv

Abstract

The rapid growth of online video platforms has outpaced the capabilities of manual content moderation, necessitating robust automated solutions. However, current AI filters often struggle with the nuance and multimodality of "questionable content." This paper proposes a strategic roadmap for developing a large-scale, scene-level multimodal dataset. By integrating visual, auditory, and contextual cues, the proposed framework aims to provide the necessary infrastructure for next-generation, context-aware moderation tools.

1. Introduction

The explosion of online streaming has created a volume of content that is impossible to screen manually. Existing automated filters are frequently brittle, focusing on single modalities—such as text or static images—and are easily circumvented by nuanced or multi-layered content. "Questionable content" is inherently subjective and context-dependent, ranging from explicit violence to subtle sarcasm or culturally specific taboos. To address these challenges, there is a critical need for a comprehensive, large-scale repository that captures the full spectrum of potentially harmful material across all video components.

2. A Multimodal Annotation Framework

The core of the proposed strategy is a move toward granular, scene-level segmentation rather than whole-video labeling. This allows for more precise training data. The framework categorizes content across four distinct modalities:

  1. Dialogue: Transcribed speech and its intent.
  2. Sound: Non-speech audio, such as screams or explosions.
  3. Soundtrack: Musical mood and lyrical content.
  4. Video: Visual depictions of actions, objects, and environments.

2.1. Defining the Taxonomy

A standardized taxonomy is essential for consistent labeling. The authors have developed detailed definitions for various categories of questionable content to ensure annotator alignment.

Proposed labels and definitions for questionable content in dialogue.

As shown in the dialogue taxonomy above, categories like "Gory Humor" and "Sarcasm" require understanding intent, which is often missed by simple keyword filters. Similarly, visual depictions must be categorized by severity and nature to distinguish between fantasy and realistic harm.

Taxonomy of visual violence and gore labels for video content.

3. The Role of Context and Subjectivity

Content cannot be judged in isolation. The framework proposes "Contextual Layering," which includes:

  • Viewer Context: Utilizing user comments and interaction history to gauge sentiment and community reaction.
  • Participant Context: Identifying the age and relationships of individuals in the video to detect bullying or inappropriate interactions.

Furthermore, recognizing that tolerance levels vary across demographics, the paper suggests using numeric scales for severity rather than binary labels. This allows platform moderators or end-users to set personalized thresholds for content filtering based on their specific needs.

4. Ethical Considerations in Data Labeling

Creating such a repository involves significant ethical risks, particularly regarding the psychological well-being of human annotators exposed to toxic content. The paper advocates for an "Annotator-Centric Design," which includes:

  • Distributing the workload across a large, diverse pool of annotators to prevent burnout.
  • Selecting annotators with relevant educational backgrounds (e.g., psychology or social work).
  • Implementing interactive tools that use AI to pre-screen or "blur" extreme content until necessary for verification.

5. Conclusion and Future Directions

This white paper serves as a blueprint for a living repository that must evolve alongside online trends. While challenges such as cultural bias and the resource-intensive nature of scene-level annotation remain, the proposed framework offers a path toward more sophisticated moderation. Future work will focus on identifying "pro-social" content to promote positive media and hosting community "shared tasks" to accelerate the development of multimodal AI models.

Summary updated

Abstract

This white paper presents a summary of the discussions regarding critical considerations to develop an extensive repository of online videos annotated with labels indicating questionable content. The main discussion points include: 1) the type of appropriate labels that will result in a valuable repository for the larger AI community; 2) how to design the collection and annotation process, as well as the distribution of the corpus to maximize its potential impact; and, 3) what actions we can take to reduce risk of trauma to annotators.
BibTeX
@misc{solorio2021white,
  author = {Thamar Solorio and Mahsa Shafaei and Christos Smailis and Mona T. Diab and Theodore Giannakopoulos and Heng Ji and Yang Liu and Rada Mihalcea and Smaranda Muresan and Ioannis A. Kakadiaris},
  title = {{White Paper: Challenges and Considerations for the Creation of a Large Labelled Repository of Online Videos with Questionable Content}},
  year = {2021},
  eprint = {2101.10894},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.10894},
}
Figure