Towards Characterizing Coordinated Inauthentic Behaviors on YouTube

Baris Kirdemir, Oluwaseyi Adeliyi, Nitin AgarwalROMCIR@ECIR 2022 · OpenReview

TL;DR

This paper introduces a novel, multi-layered approach to detect and characterize coordinated inauthentic behaviors on YouTube, moving beyond individual account analysis to focus on channel-level activity. By combining time-series engagement analysis with network structure analysis of co-commenter interactions, the researchers identify distinct patterns associated with manipulative campaigns, offering a more robust method for understanding and mitigating their spread.

What Problem Does This Paper Address?

The proliferation of coordinated inauthentic behaviors (CIBs) on social media platforms poses a significant threat to information integrity and public discourse. While much research has focused on detecting CIBs on platforms like Twitter, YouTube, a dominant platform for video sharing and information dissemination, remains less explored. Existing methods often focus on individual bot detection or specific tactics, lacking a comprehensive framework to understand and characterize large-scale, coordinated manipulation campaigns on YouTube. This gap is critical for researchers, policymakers, and journalists seeking to combat the spread of disinformation and influence operations.

Methods

The authors propose a two-pronged methodology:

  1. Engagement Trend Analysis: This involves a multi-step pipeline:
  2. Rolling Window Correlation Analysis: Examining correlations between key engagement metrics (views, subscribers, comments, video postings) over 100-day windows to identify unusual patterns (e.g., increasing views with decreasing subscribers).
  3. Anomaly Detection: Using Long Short-Term Memory (LSTM) networks to identify deviations from normal engagement patterns, generating anomaly confidence scores.
  4. Peak Detection: Applying smoothing and peak detection algorithms to the anomaly scores to pinpoint significant anomalous periods.
  5. Rule-based Classification: Developing a suspicion score for each engagement indicator based on its correlation with inauthentic behavior, then aggregating these scores with assigned weights to create an overall channel suspicion score.
  6. Principal Component Analysis (PCA): Reducing the dimensionality of the anomaly detection output to visualize channel clusters.
  7. Unsupervised Clustering (DBSCAN): Automatically identifying groups of channels with similar engagement patterns.
  8. Network Structure Analysis: This focuses on co-commenter networks:
  9. Co-commenter Network Construction: Building networks where nodes are commenters and edges represent co-commenting on the same video, with a threshold to filter out infrequent interactions.
  10. Network Feature Engineering: Extracting graph measures (degree, nodes, edges, clustering coefficient, modularity) and clique-based features (maximal cliques, average clique degree, median clique size) to capture group-level coordination.
  11. Feature Similarity Analysis: Using Pearson correlation to understand relationships between network features.
  12. Unsupervised Clustering (PCA + k-means, Hierarchical Clustering): Grouping channels based on their network structures to identify clusters exhibiting coordinated activity.

Datasets

The experimental dataset comprises 39 YouTube channels spanning November 2017 to July 2021, covering topical categories like news, defense, education, and entertainment. This includes a mix of popular news sources, football clubs, and channels previously identified as engaging in geopolitical influence campaigns. The dataset contains:

  • 936,247 videos
  • 99,415,476 comments
  • 21,067,211 unique commenters
  • Over 51 billion views
  • Over 115 million subscribers

The authors do not explicitly state they are introducing a new benchmark dataset but rather utilize a collected dataset for their analysis.

Key Insights and Ideas

  • Peak Magnitude vs. Frequency: Suspicious channels tend to exhibit fewer but higher magnitude peaks in their anomaly patterns compared to less suspicious channels. This suggests that inauthentic activity might manifest as more pronounced, albeit less frequent, spikes in engagement.
  • Co-commenter Networks as Implicit Artifacts: Analyzing the network of commenters provides a powerful way to infer coordination. Tightly knit sub-networks (cliques) and high ratios of commenters participating in cliques signal strong coordination among users.
  • Engagement Metrics as Indicators: While multiple engagement metrics are analyzed, views and subscriber counts appear to be particularly relevant for identifying inauthentic growth tactics.
  • Unsupervised Clustering Effectiveness: Both engagement trend analysis (PCA + DBSCAN) and network feature analysis (PCA + k-means) successfully identified clusters of channels with similar characteristics, demonstrating the utility of unsupervised methods for detecting coordinated behaviors.
  • Platform Specificity: The study highlights the need for YouTube-specific approaches due to its unique platform architecture and the "black-box" nature of its algorithms, which can obscure coordinated activities.

Key Results

  • The study successfully identified clusters of channels exhibiting similar engagement trends and network structures.
  • Channels with high suspicion scores were found to have a relatively lesser number of anomaly peaks, but these peaks were of higher magnitude.
  • Network analysis revealed that channels with a higher number of nodes (commenters) forming cliques often signal strong coordination.
  • The unsupervised clustering approaches (DBSCAN for engagement, k-means/hierarchical for network features) yielded comparable cluster configurations, validating their effectiveness.
  • A specific example highlighted a channel with a massive subscriber surge that was later corrected by YouTube, underscoring the dynamic nature of inauthentic activity and the potential for detection.

Limitations and Future Work

  • Temporal Limitations in Network Analysis: The current network analysis does not incorporate temporal limitations, meaning it doesn't track how co-commenter networks evolve over time. Future work will explore features extracted from network activity within limited timeframes.
  • Validation Set Challenges: Obtaining a reliable validation set of inauthentic channels is challenging, as such channels are often taken down without public explanation. This necessitated a rule-based classification approach with manual annotation.
  • Combining Methodologies: The authors aim to combine their engagement trend and network structure analysis into a unified framework for a more comprehensive detection of suspicious behavior.
  • Improving Accuracy and Timeliness: The ultimate goal is to develop an accurate and effective methodology for the timely detection of harmful manipulative campaigns on YouTube.

Summary updated

BibTeX
@inproceedings{kirdemir2022towards,
  author = {Baris Kirdemir and Oluwaseyi Adeliyi and Nitin Agarwal},
  title = {{Towards Characterizing Coordinated Inauthentic Behaviors on YouTube}},
  booktitle = {ROMCIR@ECIR},
  year = {2022},
  url = {https://openreview.net/forum?id=KwXKLEdiQe},
}
Figure