EX-FIQA: Leveraging Intermediate Early eXit Representations from Vision Transformers for Face Image Quality Assessment

Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Jan Niklas Kolf, Andrea Atzori, Fadi Boutros, Naser DamerarXiv 2026 · arXiv

TL;DR

This paper challenges the conventional wisdom that only the final layer of a Vision Transformer (ViT) matters for Face Image Quality Assessment (FIQA). By systematically exploring intermediate layers, the authors introduce EX-FIQA, a framework that leverages early network exits to cut computational costs by up to 50% and uses a depth-weighted score fusion strategy to outperform traditional final-layer-only approaches.

What Problem Does This Paper Address?

Face Image Quality Assessment (FIQA) is a crucial component of biometric systems. Unlike standard image quality metrics that look at aesthetics (like blur or noise), FIQA measures "biometric utility"—determining if a facial image contains enough clear information to be successfully matched by an automated face recognition system.

Recently, Vision Transformers (ViTs) have become the state-of-the-art backbone for FIQA. However, existing methods rely exclusively on the representations extracted from the very last layer of the network. This design choice has two major drawbacks:

  1. Missed Signals: It ignores valuable, complementary quality cues (like low-level textures or mid-level geometric alignments) captured at intermediate depths.
  2. High Computational Cost: Running a full 12-block ViT is computationally expensive, making it difficult to deploy on resource-constrained edge devices or in real-time surveillance systems where poor-quality frames need to be filtered out as fast as possible.

Methods

The authors introduce EX-FIQA, a training-free framework that probes the intermediate layers of pre-trained ViT-FIQA models.

A unique architectural property of ViTs is that they maintain a constant token dimensionality ($D = 512$) across all transformer blocks. The authors exploit this structural consistency to apply the original, pre-trained final-layer quality regression head directly to any of the 12 intermediate blocks without modifying the architecture or performing additional training.

The paper evaluates two main execution paradigms:

  1. Early Exits (EX-FIQA): Halting inference at an intermediate block (e.g., block 6 or 7) to save computational power.
  2. Score Fusion (EX-FIQA-FW): Running the full network but combining the quality scores from all intermediate blocks using a depth-based linear weighting scheme ($w_l \propto l$). This scheme assigns progressively higher importance to deeper blocks, capturing the hierarchical nature of the features.

Datasets

The framework was extensively evaluated across eight standard face recognition benchmark datasets:

  • LFW, AgeDB-30, CFP-FP, CALFW, Adience, CPLFW, XQLFW, and IJB-C.

To ensure the quality scores generalize well, the authors conducted cross-model evaluations using four distinct, state-of-the-art convolutional face recognition models: ArcFace, ElasticFace, MagFace, and CurricularFace. No new datasets were introduced in this work.

Key Insights and Ideas

  • Complementary Feature Maps: Attention map analysis revealed that early transformer blocks focus on low-level textures and edges, middle layers capture object parts, and deeper layers encode high-level semantic structures. Both low- and high-level features provide distinct, useful signals for quality assessment.
  • The "Good Enough" Middle: Features mature surprisingly early in the network. For many datasets, exiting around the middle layers (blocks 6–10) yields quality predictions that are highly competitive with the final layer.
  • Parameter Reusability: Because token dimensions are invariant across ViT layers, intermediate layers can reuse the final layer's regression parameters out-of-the-box, enabling zero-shot early exits.

Key Results

  • Superior Accuracy via Fusion: The depth-weighted fusion strategy (EX-FIQA-FW) consistently outperformed single-exit final-layer models, achieving top-1 performance on highly challenging, large-scale datasets like IJB-C and Adience.
  • Massive Computational Savings: For resource-constrained deployments, exiting early at block 6 or 7 provided up to 50% computational savings (GFLOPs) with negligible impact on quality assessment accuracy.
  • Actionable Operating Points: The authors established clear deployment recommendations based on system constraints:
  • Maximum Efficiency: Exit 6 (50% compute savings).
  • Balanced Performance: Exit 7 (41% compute savings).
  • Maximum Accuracy: EX-FIQA-FW fusion.

Limitations and Future Work

The primary limitation acknowledged by the authors is that this study evaluates early exits strictly at inference time on a frozen, pre-trained model.

For future work, they suggest exploring active training strategies to further bridge the accuracy gap between early and late exits. This includes implementing specialized intermediate branch architectures (like lightweight convolutional heads), using self-distillation during training so early layers explicitly learn to mimic final-layer behavior, and developing entropy-based dynamic routing policies to automatically determine the optimal exit point on an image-by-image basis.

Summary updated

Abstract

Face Image Quality Assessment is crucial for reliable face recognition systems, yet existing Vision Transformer-based approaches rely exclusively on final-layer representations, ignoring quality-relevant information captured at intermediate network depths. This paper presents the first comprehensive investigation of how intermediate representations within ViTs contribute to face quality assessment through early exit mechanisms and score fusion strategies. We systematically analyze all twelve transformer blocks of ViT-FIQA architectures, demonstrating that different depths capture distinct and complementary quality-relevant information, as evidenced by varying attention patterns and performance characteristics across network layers. We propose a score fusion framework that combines quality predictions from multiple transformer blocks without architectural modifications or additional training. Our early exit analysis reveals optimal performance-efficiency trade-offs, enabling significant computational savings while maintaining competitive performance. Through extensive evaluation across eight benchmark datasets using four FR models, we demonstrate that our fusion strategy improves upon single-exit approaches. Our proposed quality fusion approach employs depth-weighted averaging that assigns progressively higher importance to deeper transformer blocks, achieving the best quality assessment performance by effectively leveraging the hierarchical nature of feature learning in ViTs. Our work challenges the conventional wisdom that only deep features matter for face analysis, revealing that intermediate representations contain valuable information for quality assessment. The proposed framework offers practical benefits for real-world biometric systems by enabling adaptive computation based on resource constraints while maintaining competitive quality assessment capabilities.
BibTeX
@misc{ozgur2026ex,
  author = {Guray Ozgur and Tahar Chettaoui and Eduarda Caldeira and Jan Niklas Kolf and Andrea Atzori and Fadi Boutros and Naser Damer},
  title = {{EX-FIQA: Leveraging Intermediate Early eXit Representations from Vision Transformers for Face Image Quality Assessment}},
  year = {2026},
  eprint = {2604.22842},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.22842},
}

Loading related content...

Figure