Interactive Event Sifting using Bayesian Graph Neural Networks

José Nascimento, Nathan Jacobs, Anderson Rocha

Abstract

Forensic analysts often use social media imagery and texts to understand important events. A primary challenge is the initial sifting of irrelevant posts. This work introduces an interactive process for training an event-centric, learning-based multimodal classification model that automates sanitization. We propose a method based on Bayesian Graph Neural Networks (BGNNs) and evaluate active learning and pseudo-labeling formulations to reduce the number of posts the analyst must manually annotate. Our results indicate that BGNNs are useful for social-media data sifting for forensics investigations of events of interest, the value of active learning and pseudo-labeling varies based on the setting, and incorporating unlabelled data from other events improves performance.
BibTeX
@inproceedings{nascimento2024interactive,
  author = {José Nascimento and Nathan Jacobs and Anderson Rocha},
  title = {{Interactive Event Sifting using Bayesian Graph Neural Networks}},
  booktitle = {WIFS},
  year = {2024},
  doi = {10.1109/wifs61860.2024.10810718},
}
Figure