Surveillance Facial Image Quality Assessment: A Multi-dimensional Dataset and Lightweight Model

Yanwei Jiang, Wei Sun, Yingjie Zhou, Xiangyang Zhu, Yuqin Cao, Jun Jia, Yunhao Li, Sijing Wu, Dandan Zhu, Xingkuo Min, Guangtao Zhai

This paper, recently published in IEEE Transactions on Circuits and Systems for Video Technology, introduces a new framework for assessing the quality of facial images captured by surveillance systems. The authors present SFIQA-Bench, a comprehensive multi-dimensional dataset, and SFIQA-Assessor, a lightweight, multi-task deep learning model designed for real-time deployment on edge devices.

TL;DR

The researchers developed a benchmark and an efficient multi-task model to evaluate surveillance facial image quality across six dimensions, crucially including "fidelity" to ensure identity preservation. Their lightweight model, SFIQA-Assessor, achieves state-of-the-art performance with significantly lower computational costs than existing heavy-duty alternatives.

What Problem Does This Paper Address?

Surveillance facial images (SFIs) are notoriously difficult to process. Captured in unconstrained environments, they suffer from low resolution, motion blur, and poor lighting. While modern face restoration techniques (like those based on GANs or Diffusion models) can make these images look much better to the human eye, they often introduce "hallucinated" details—pseudo-structures that don't belong to the actual person.

This creates a conflict: a restored image might look "high quality" but lose its fidelity, meaning it no longer accurately represents the individual's identity. Existing Facial Image Quality Assessment (FIQA) methods typically focus on either perceptual beauty or recognition utility, but rarely both. This paper bridges that gap by treating quality as a multi-dimensional problem where identity preservation is just as important as visual clarity.

Fig. 1. (a) High-quality reference facial images. (b) Corresponding degraded versions generated by applying JPEG compression and blur. (c) and (d) Restored images produced by two representative face restoration methods [2], [3]. While the restorations enhance perceptual quality, they may introduce hallucinated textures or pseudo-structures, potentially compromising identity fidelity.

Methods

The authors propose SFIQA-Assessor, a lightweight model built on the EdgeNeXt backbone. Its architecture features several novel components:

  1. Multi-View Input: Instead of just looking at the whole image, the model processes three complementary views: the original loosely cropped image (context), a tightly cropped face region, and a specific eyes-and-mouth region.
  2. Facial-Aware Encoder: It uses a hierarchical multi-scale feature extraction strategy. Shallow layers capture low-level features like noise and sharpness, while deeper layers capture high-level semantic information needed to judge fidelity.
  3. Cross-View Feature Fusion: A lightweight attention module fuses these views, using a low-rank approach to keep computational complexity linear rather than quadratic.
  4. Task-Aware Decoder: To handle the six different quality dimensions, the model uses learnable "task tokens." These tokens interact with the fused facial features through a two-stage self-attention and cross-attention process, allowing the model to disentangle specific quality cues (e.g., separating noise from sharpness).

Datasets

The study introduces SFIQA-Bench, the first multi-dimensional benchmark tailored for surveillance.

  • Scale: 5,004 real-world images.
  • Sources: Captured by three different professional surveillance camera models across three scenarios: indoor, outdoor pedestrian, and in-vehicle (ITS).
  • Annotations: 750,600 subjective ratings from 100 participants.
  • Dimensions: Each image is rated on six scales: Noise, Sharpness, Colorfulness, Contrast, Fidelity, and Overall Quality.

The authors also validated their model on existing perceptual FIQA datasets, GFIQA-20K and CGFIQA-40K, to demonstrate generalization.

Key Insights and Ideas

  • The Fidelity Gap: The paper highlights a significant misalignment between recognition quality and perceptual quality. An image that a machine can easily recognize isn't always one a human would call "high quality," and vice versa.
  • Dominant Factors: Through multiple linear regression, the authors found that sharpness and fidelity are the most significant contributors to a human's perception of "overall quality" in a surveillance context.
  • Scenario Impact: Images from vehicle-based surveillance (driver monitoring) consistently show lower quality scores due to extreme angles, reflections, and motion blur compared to standard pedestrian surveillance.
  • Efficiency Matters: By using EdgeNeXt and a task-token-based decoder, the authors show that you don't need a massive Transformer to get SOTA results. Their "XXS" variant is extremely tiny (5.28M parameters) yet remains highly competitive.
There is often a misalignment between recognition quality and perceptual quality; high recognition utility does not guarantee high visual quality.

Key Results

  • Performance: SFIQA-Assessor outperformed almost all general IQA and FIQA methods across all six dimensions.
  • Efficiency vs. SOTA: Compared to the previous top-performer, DSL-FIQA, the SFIQA-Assessor-S achieved nearly identical SRCC (0.9379 vs 0.9411) but with:
  • 10x fewer parameters (17.76M vs 252.69M).
  • 600x fewer MACs (3.75G vs 2427.2G).
  • 40x lower latency (~10ms vs ~518ms).
  • Generalization: In cross-dataset tests, models trained on SFIQA-Bench generalized well to other datasets, whereas models trained on cleaner Internet-sourced datasets struggled with the complex distortions of SFIQA-Bench.

Limitations and Future Work

The authors note that the sharpness dimension currently acts as a "catch-all" for various clarity-related issues, including motion blur, defocus, and low resolution. They suggest that future work should attempt to further disentangle these specific types of degradation to provide even more granular feedback for camera systems. Additionally, while the model is lightweight, further optimization for specific NPU architectures in surveillance cameras could be explored.

Summary updated

Abstract

Surveillance facial images are often captured under unconstrained conditions, resulting in severe quality degradation due to factors such as low resolution, motion blur, occlusion, and poor lighting. Although recent face restoration techniques applied to surveillance cameras can significantly enhance visual quality, they often compromise fidelity (i.e., identity-preserving features), which directly conflicts with the primary objective of surveillance images -- reliable identity verification. Existing facial image quality assessment (FIQA) predominantly focus on either visual quality or recognition-oriented evaluation, thereby failing to jointly address visual quality and fidelity, which are critical for surveillance applications. To bridge this gap, we propose the first comprehensive study on surveillance facial image quality assessment (SFIQA), targeting the unique challenges inherent to surveillance scenarios. Specifically, we first construct SFIQA-Bench, a multi-dimensional quality assessment benchmark for surveillance facial images, which consists of 5,004 surveillance facial images captured by three widely deployed surveillance cameras in real-world scenarios. A subjective experiment is conducted to collect six dimensional quality ratings, including noise, sharpness, colorfulness, contrast, fidelity and overall quality, covering the key aspects of SFIQA. Furthermore, we propose SFIQA-Assessor, a lightweight multi-task FIQA model that jointly exploits complementary facial views through cross-view feature interaction, and employs learnable task tokens to guide the unified regression of multiple quality dimensions. The experiment results on the proposed dataset show that our method achieves the best performance compared with the state-of-the-art general image quality assessment (IQA) and FIQA methods, validating its effectiveness for real-world surveillance applications.
BibTeX
@article{jiang2026surveillance,
  author = {Yanwei Jiang and Wei Sun and Yingjie Zhou and Xiangyang Zhu and Yuqin Cao and Jun Jia and Yunhao Li and Sijing Wu and Dandan Zhu and Xingkuo Min and Guangtao Zhai},
  title = {{Surveillance Facial Image Quality Assessment: A Multi-dimensional Dataset and Lightweight Model}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2026},
  doi = {10.1109/tcsvt.2026.3662100},
}
Figure