PV-VTT: A Privacy-Centric Dataset for Mission-Specific Anomaly Detection and Natural Language Interpretation

Ryozo Masukwa, Sanggeon Yun, Yoshiki Yamaguchi, Mohsen Imani

Abstract

Traditional Video Anomaly Detection (VAD) often focuses on reactive responses to violent crimes. This paper introduces PV-VTT, a novel multimodal dataset designed for proactive detection of "precursor" privacy violations such as trespassing and voyeurism. Alongside the dataset, we propose a Hierarchical Graph Neural Network (GNN) framework that identifies "reasoning paths" to guide Large Language Models (LLMs) in generating video descriptions. Our approach achieves a mean Area Under the Curve (mAUC) of 94.61 while reducing LLM token consumption by approximately 8,000 tokens per inference compared to standard methods.

Introduction

Current surveillance AI typically alerts authorities after a significant incident—like a robbery or assault—has occurred. However, serious crimes are frequently preceded by subtle privacy violations: scouting a perimeter, unauthorized drone surveillance, or "Peeping Tom" behavior. Detecting these "pre-crime" indicators is challenging due to their context-dependent nature. To address this, we present PV-VTT, the first dataset to combine mission-specific privacy violation labels with natural language descriptions.

The PV-VTT Dataset

PV-VTT comprises 702 videos totaling 31 hours of footage (2.3 million frames). It covers five distinct categories: Drone Trespassing, Peeping Tom, Pervert, Trespassing, and Unauthorized Photographing.

Dataset Construction and Privacy

To maintain the privacy of individuals in the source footage, PV-VTT does not release raw video. Instead, it provides high-dimensional feature vectors extracted via ImageBind. The annotation process involves a hybrid LLM-human pipeline where frame-level anomalies are identified, and text descriptions are anonymized to remove sensitive identifiers.

The PV-VTT data collection and annotation pipeline, featuring frame-level labeling and LLM-based text anonymization.

Dataset Characteristics

The dataset provides a diverse temporal distribution of events, allowing models to learn the varying durations and patterns associated with different types of privacy violations.

Distribution of video durations across the five primary privacy violation categories in the PV-VTT dataset.

Methodology: Mission-Specific Reasoning

We propose a two-stage framework for efficient anomaly interpretation:

  1. Hierarchical GNN: We construct a mission-specific Knowledge Graph (KG) using GPT-4o and ConceptNet. A GNN processes video features through this graph to classify the violation.
  2. Impactive Reasoning Paths: The model identifies the most influential nodes in the KG that led to a classification (e.g., linking "sensor" to "touch" to "pervert").
  3. Efficient LLM Prompting: Instead of processing entire video streams, we send only the single most representative frame and the textual reasoning paths to an LLM. This provides sufficient semantic context for high-quality captioning at a fraction of the computational cost.

Results

Our MissionGNN model demonstrates superior performance on the PV-VTT benchmark, achieving an mAUC of 94.61. Beyond detection accuracy, the reasoning-path approach offers significant operational advantages. By substituting raw video frames with structured text paths, we reduced the average prompt size from ~9,500 tokens to just 1,123 tokens. Human evaluation confirmed that these compressed prompts do not sacrifice descriptive quality, with an 80% approval rating for the generated captions.

Conclusion

PV-VTT and the associated GNN-LLM framework represent a shift toward proactive, privacy-preserving surveillance. By focusing on precursor anomalies and structural reasoning, we can build AI systems that are both more effective at crime prevention and more cost-efficient to deploy. Future research will focus on refining the alignment between news-derived manuscripts and visual ground truth.

Summary updated

Abstract

Video crime detection is a significant application of computer vision and artificial intelligence. However, existing datasets primarily focus on detecting severe crimes by analyzing entire video clips, often neglecting the precursor activities (i.e., privacy violations) that could potentially prevent these crimes. To address this limitation, we present PV-VTT (Privacy Violation Video To Text), a unique multimodal dataset aimed at identifying privacy violations. PV-VTT provides detailed annotations for both video and text in scenarios. To ensure the privacy of individuals in the videos, we only provide video feature vectors, avoiding the release of any raw video data. This privacy-focused approach allows researchers to use the dataset while protecting participant confidentiality. Recognizing that privacy violations are often ambiguous and context-dependent, we propose a Graph Neural Network (GNN)-based video description model. Our model generates a GNN-based prompt with image for Large Language Model (LLM), which deliver cost-effective and high-quality video descriptions. By leveraging a single video frame along with relevant text, our method reduces the number of input tokens required, maintaining descriptive quality while optimizing LLM API-usage. Extensive experiments validate the effectiveness and interpretability of our approach in video description tasks and flexibility of our PV-VTT dataset.
BibTeX
@inproceedings{masukwa2024pv,
  author = {Ryozo Masukwa and Sanggeon Yun and Yoshiki Yamaguchi and Mohsen Imani},
  title = {{PV-VTT: A Privacy-Centric Dataset for Mission-Specific Anomaly Detection and Natural Language Interpretation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure