TL;DR
Q-Router is an agentic framework for universal Video Quality Assessment (VQA) that uses a Vision-Language Model (VLM) as an adaptive orchestrator to dynamically select and fuse specialized expert models (e.g., UGC and AIGC evaluators). By coupling a budget-aware, multi-tier routing architecture with a spatio-temporal artifact localization pipeline, Q-Router delivers state-of-the-art accuracy, robust cross-domain generalization, and interpretable visual evidence of video distortions.
What Problem Does This Paper Address?
Traditional deep learning models for video quality assessment directly regress a single scalar score from raw pixels. While effective within their specific training domains, they suffer from three major shortcomings:
- Poor Cross-Domain Generalization: Video formats have expanded from user-generated content (UGC) to AI-generated content (AIGC). Standard UGC-trained evaluators degrade sharply when facing AIGC artifacts (e.g., a ~70% drop in PLCC for COVER).
- Limited Interpretability: Scalar quality ratings function as black boxes, failing to indicate why or where quality degrades.
- Inflexibility: Incorporating new distortion types or content classes typically requires complete retraining or architectural redesigns.
Methods
Q-Router addresses these challenges via an agentic, multi-tiered framework:
[Input Video]
│
▼
[VLM Router (e.g., GPT-4o)] ──> Classifies Domain & Allocates Weights
│
├──> Tier 0: Single-Expert Fast Path (Coarse inference)
├──> Tier 1: Multi-Expert Weighted Fusion (Dynamic weighting)
└──> Tier 2: Fine-Grained Artifact Localization Pipeline
- Agentic Routing & Weighted Fusion: A VLM acts as an orchestrator. It evaluates sampled frames and metadata to classify the video (UGC, AIGC, or Computer Graphics), queries specialized expert models from an extensible pool (including COVER, DOVER, UVQ, MaxVQA, ModularBVQA, and T2VQA), and dynamically computes fused scores based on model-task fit and inter-model agreement.
- Three-Tier Compute Hierarchy:
- Tier 0: Routes to a single best-fit expert for minimal latency.
- Tier 1: Blends predictions across multiple experts using relevance-aware dynamic weighting.
- Tier 2: Augments multi-expert fusion with explicit spatio-temporal artifact localization.
- Spatio-Temporal Artifact Localization (Tier 2):
- Probabilistic Extraction: Identifies suspicious candidate frames using handcrafted indicators (motion residuals, sharpness energy, gradient kurtosis, edge density, and HSV histogram shifts) grouped by hysteresis thresholding.
- VLM Semantic Filtering: GPT-4o filters false alarms, keeping only frames exhibiting hallucinations, image artifacts, or AI inconsistencies.
- Perceptual Heatmap Generation: Successive frames are motion-compensated via optical flow, and spatial LPIPS calculates pixel-level difference maps highlighting local distortions.
Datasets
The authors evaluated Q-Router across diverse benchmarks:
- UGC VQA Benchmarks: KoNViD-1k, LSVQ-1080, LSVQ-Test, LIVE-VQC, and YouTube-UGC (YT-UGC).
- AIGC VQA Benchmark: T2VQA-DB (focused on text-to-video distortions).
- Video Quality Question Answering: Q-Bench-Video (both development and test splits).
Key Insights and Ideas
- Expert Ensembling Solves the UGC–AIGC Divide: Standalone models fail on out-of-domain distributions (e.g., DOVER achieves only 0.0642 PLCC on AIGC, while the AIGC model T2VQA manages just 0.1717 PLCC on KoNViD-1k). Routing by content semantics bridges this gap without retraining underlying models.
- System 1 vs. System 2 Reasoning in VLM Scoring: Direct zero-shot quality prompting of VLMs correlated surprisingly well with human Mean Opinion Scores (MOS), whereas forced Chain-of-Thought (CoT) reasoning degraded performance—suggesting deliberative text reasoning can misalign with human perceptual intuition. Incorporating specialized expert predictions restored and surpassed this baseline.
- Modular Test-Time Scaling: Instead of end-to-end black-box diagnostic models, combining classical motion-compensated perceptual features with VLM filtering creates reliable, model-agnostic localization heatmaps.
Key Results
- Overall VQA Performance: Across both UGC and AIGC benchmarks, Q-Router (Tier 1) achieved the highest overall average performance (0.8539 PLCC / 0.8407 SRCC), outperforming all standalone baselines by over 20%.
- AIGC Robustness: On T2VQA-DB, Q-Router attained 0.8283 PLCC, drastically exceeding conventional UGC models (PLCC < 0.25).
- Video QA SOTA: On the Q-Bench-Video benchmark:
- Dev Split: Tier 2 achieved 60.07% overall accuracy, outperforming proprietary GPT-4o (56.91%).
- Test Split: Tier 1 scored 59.45% vs. GPT-4o’s 56.50%, with particularly large margins on AIGC-related questions (64.42% vs. 53.80%).
- Model Agnostic: Ablation with an open-source 4B model (Qwen-VL) maintained competitive performance and a steady upward trend from Tier 0 (50.06%) to Tier 2 (51.42%), verifying the architecture's generalizability.
Limitations and Future Work
- Inference Latency: VLM routing and optical-flow preprocessing introduce substantial computational overhead (routing latency ranges from 7.11s in Tier 0 to 16.36s in Tier 2 per video, excluding expert execution time), making higher tiers unsuited for real-time video streaming.
- Dataset-Specific Peak Performance: On certain isolated benchmarks, the top specialized expert still slightly edges out the fused model (e.g., UVQ on YT-UGC), as Q-Router optimizes for cross-domain stability rather than single-dataset tuning.
- Future Directions: Developing fully end-to-end, learned diagnostic localization modules to replace handcrafted feature stages, and investigating Q-Router's localized heatmaps as reward functions for post-training video generation models.
Summary updated