VENUS: A VLLM-driven Video Content Discovery System for Real Application Scenarios

Minyi Zhao, Yi Liu, Jianfeng Wen, Boshen Zhang, Hailang Chang, Zhiheng Ouyang, Jie Wang, Wensong He, Shuigeng ZhouEMNLP 2025 · OpenReview

TL;DR

This paper introduces VENUS, a novel Vision-Language Large Model (VLLM) system designed for Video Content Discovery (VCD). VENUS addresses the limitations of existing VCD methods by leveraging VLLMs to understand complex policies and video content, offering not just a "yes/no" answer but also a detailed reasoning process and a confidence score. The authors also introduce VCD-Bench, a large-scale, real-world dataset for VCD evaluation.

What Problem Does This Paper Address?

The internet is flooded with videos, making it crucial to identify specific content based on predefined text policies. This "Video Content Discovery" (VCD) is vital for maintaining healthy online ecosystems by filtering out harmful or irrelevant content. Traditional VCD approaches, like multiple classifiers or similarity-based systems, struggle with managing a growing number of policies, lack generalization, and often perform poorly. Existing VLLMs, while powerful, are trained on general web content and often fail to handle the nuances of rare video content and the dynamic nature of VCD policies. This gap highlights the need for a more robust and adaptable VCD system.

Methods

VENUS tackles this challenge with a two-pronged approach:

  1. Automatic Policy-Guided Sequential Annotator (APSA): To overcome the scarcity of high-quality, VCD-specific training data, VENUS introduces APSA. This novel data engine generates instruct-tuning data by guiding a VLLM through a step-by-step reasoning process. It includes a quality evaluation phase to ensure the generated data is accurate and useful, addressing issues like incorrect initial judgments or VLLM misinterpretations.
  2. Extended VLLM Inference: VENUS modifies the standard VLLM inference process to better suit VCD. It not only generates a textual response (e.g., "Yes/No") but also provides a confidence score for its decision. This score is crucial for downstream applications like threshold adjustments and filtering. The system also incorporates a dynamic high-resolution module to effectively process video content and a projector to align visual and textual features.

Datasets

  • VCD-Bench: A newly constructed, large-scale dataset specifically for VCD. It comprises 13 distinct text policies and 57,000 videos, designed to reflect real-world VCD scenarios and densities.
  • Public VLLM Evaluation Datasets: VENUS was also evaluated on several general VQA (Visual Question Answering) benchmarks, including TextVQA, MME, SEED-Bench, and MM-Vet, to demonstrate its broader capabilities.

Key Insights and Ideas

  • Reasoning is Key: Simply outputting "Yes/No" is insufficient for VCD. Providing a step-by-step reasoning process, explaining why a video matches or doesn't match a policy, is critical for trust and debugging.
  • Data Quality Over Quantity: The APSA's quality evaluation phase is crucial. Simply generating more data without ensuring its accuracy can be detrimental to model performance, especially for rare VCD policies.
  • Confidence Matters: A confidence score is essential for practical VCD systems, allowing for flexible decision-making and integration into larger content moderation pipelines.
  • VCD and General VQA are Correlated: The paper suggests a positive correlation between a model's general VQA ability and its VCD performance, implying that a strong foundation in understanding visual and textual information benefits VCD.

Key Results

VENUS demonstrated superior performance on the VCD-Bench dataset, achieving significantly higher Average Precision (AP) and Recall at Precision=0.5 ([email protected]) compared to existing state-of-the-art methods, including specialized VCD systems and general VLLMs. For instance, VENUS achieved an AP of 0.565 and [email protected] of 0.573, outperforming LLaVA-OV (0.430/0.475) and LLaVA-Next (0.202/0.153). On general VQA benchmarks, VENUS also showed competitive results, winning 7 out of 10 tasks. The system was successfully deployed in three simulation scenarios, including a real-time running system, a zero-shot retrieval task, and a few-shot scenario using LoRA, showcasing its practical applicability.

Limitations and Future Work

The primary limitation identified is VENUS's efficiency, with a query-per-second (QPS) of 5, which is lower than traditional systems. This means VENUS is currently best suited for handling difficult "tough cases" that existing systems miss, rather than replacing them entirely. Future work will focus on improving VENUS's efficiency through techniques like distillation and inference acceleration, and further enhancing its performance using methods like Chain of Thought (CoT) and Mixture of Experts (MoE).

Summary updated

Abstract

Video Content Discovery (VCD) is to identify the specific videos defined by a certain pre-specified text policy (or constraint), which plays a crucial role in building a healthy and high-quality Web content ecology. Currently, related works typically employ multiple classifiers or similarity-based systems to support VCD. However, these approaches are difficult to manage, lack generalization power, and suffer from low performance. To tackle these problems, this paper presents a new Vision-Language Large Model (VLLM)-driven VCD system called VENUS (the abbreviation of Video contENt UnderStander). Concretely, we first develop an automatic policy-guided sequential annotator (APSA) to generate high-quality, VCD-specific, and reasoning-equipped instruct-tuning data for model training, then extend the VLLM inference to support VCD better. Following that, we construct a real VCD test set called VCD-Bench, which includes a total of 13 policies and 57K videos. Furthermore, to evaluate its practical efficacy, we deploy VENUS in three different real scenarios. Extensive experiments on both the VCD-Bench and public evaluation datasets for various VCD-related tasks demonstrate the superiority of VENUS over existing baselines.
BibTeX
@inproceedings{zhao2025venus,
  author = {Minyi Zhao and Yi Liu and Jianfeng Wen and Boshen Zhang and Hailang Chang and Zhiheng Ouyang and Jie Wang and Wensong He and Shuigeng Zhou},
  title = {{VENUS: A VLLM-driven Video Content Discovery System for Real Application Scenarios}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=H2u5X4cl5B},
}
Figure