Group-wise Scaling and Orthogonal Decomposition for Domain-Invariant Feature Extraction in Face Anti-Spoofing

Seungjin Jung, Kanghee Lee, Yonghyun Jeong, Haeun Noh, Jungmin Lee, Jongwon Choi

TL;DR

Current domain generalizable face anti-spoofing methods fail when encountering unseen environments because they only align the direction (weights) of decision boundaries while ignoring the classification thresholds (biases). This paper introduces GD-FAS, a framework that jointly aligns weights and biases using Feature Orthogonal Decomposition and Group-wise Scaling Risk Minimization. The approach achieves state-of-the-art generalization and significantly improves model calibration on unseen target domains.

What Problem Does This Paper Address?

Face Recognition (FR) systems are everywhere, from airport security to smartphone unlocking. Consequently, Face Anti-Spoofing (FAS)—detecting whether a face presented to a camera is real or a fake photo/video spoof—is a critical defense mechanism.

The real challenge lies in Domain Generalizable FAS (DGFAS): training a model on a few source domains (e.g., specific cameras, lighting conditions) so that it performs flawlessly on entirely unseen target domains.

Existing methods typically try to extract domain-invariant features by aligning the directions (classifier weights) of local decision boundaries. However, the authors point out a glaring omission: they ignore the bias terms. As shown in Figure 1, if the bias terms remain misaligned, the classification thresholds shift across domains, leading to degraded performance and poor reliability when the model faces a new environment.

Figure 1. Bias term and weight term across domains: Decision boundaries are defined by the classifier’s weights and biases. Do- main invariance maintains consistent distinguishability across do- mains, typically by sharing a common weight term. In contrast, domain specificity represents domain gaps arising from variations in bias terms. (a) Domain-wise contrastive loss enhances domain- specific separability with bias alignment but struggles to extract domain-invariant features. (b) SAFAS aligns local decision bound- aries for improved domain invariance, yet residual domain gaps persist, particularly due to bias misalignment. (c) Our approach addresses this by jointly aligning weights and biases using Group- wise Scaling (GS) and Feature Orthogonal Decomposition (FOD), establishing a unified and consistent decision boundary.

Methods

To solve this, the authors propose a framework called GD-FAS, built on top of the vision-language model CLIP. It introduces two novel components to jointly align weights and biases:

  1. Group-wise Scaling Risk Minimization (GS-RM): This module balances learning across multiple domains by adaptively scaling group-wise losses. It penalizes low-loss (easy/highly similar) groups and boosts high-loss (hard/dissimilar) groups during training, forcing the classifier to find a consistent bias threshold that works across all groups.
  2. Feature Orthogonal Decomposition (FOD): Using the classic Gram-Schmidt orthogonalization process, FOD explicitly splits the image feature space into two distinct, perpendicular subspaces: domain-invariant and domain-specific.

By ensuring complete mathematical orthogonality, the domain-specific features have an inner product of zero with the classifier weights. This prevents domain-specific variations from interfering with the aligned bias terms.

Figure 2. Framework overview: Our framework, based on CLIP [28], achieves domain generality by decomposing representations into invariant and specific components within the Text-Image embedding space. (1) GS-RM balances group-wise loss across domains to en- hance domain invariance. (2) FOD leverages the Gram-Schmidt process to decompose domain-invariant and domain-specific components, effectively mitigating major spurious correlations. (3) The feature extractor and classifier enhance the embedding of images within the space, ensuring robust feature representation.

Additionally, the authors introduce Expected Calibration Error (ECE) to the DGFAS field to quantitatively measure how reliable the model's confidence scores are across different domains.

Datasets

The framework was evaluated using a leave-one-out protocol across four widely recognized benchmark FAS datasets:

  • CASIA-FA (C)
  • OULU-NPU (O)
  • Idiap Replay-Attack (I)
  • MSU-MFSD (M)

For extended testing, they also utilized CelebA-Spoof as an extra source dataset, and validated generalization on SiW-Mv2, Surf (S), CeFA (C), and WMCA (W).

Key Insights and Ideas

  • The Geometry of Bias: If domain-specific features ($f_S$) are perfectly orthogonal to the invariant basis ($bI$), then the term $f_S \cdot w^T$ drops to zero. This mathematically guarantees that the domain bias is consistently aligned as zero across all domains.
  • Calibration as a Proxy for Alignment: A well-calibrated model (low ECE) means its predicted probabilities match actual accuracy. The authors show that tracking ECE is an excellent way to evaluate whether bias terms are truly aligned across domain shifts.
  • Stability Over Early Stopping: Many recent DGFAS methods (like FLIP) suffer from severe performance degradation if they aren't stopped early during training. GD-FAS achieves exceptional convergence stability, maintaining top-tier performance across extended training epochs.

Key Results

GD-FAS achieves state-of-the-art performance across standard benchmarks, measured by Half Total Error Rate (HTER) and Area Under the Curve (AUC):

  • Benchmark Superiority: Without extra data, GD-FAS achieves an average HTER of 1.83%, outperforming recent methods like SAFAS (3.60%) and CFPL (3.60%).
  • Perfect Predictions: When utilizing CelebA-Spoof as an extra source dataset, GD-FAS achieves a 0.00% HTER (perfect classification) in the OCI $\rightarrow$ M evaluation scenario.
  • Reliability: GD-FAS consistently yields the lowest ECE scores across almost all domain shift scenarios, proving that its threshold alignment strategy works.
Figure 4. Learning stability: Changes in performance measurements according to epoch settings for each domain.

Limitations and Future Work

The authors note a slight trade-off between model reliability (ECE) and raw generalization metrics when combining GS-RM and FOD. While using FOD alone reduces ECE to its absolute lowest levels, the full combination of GS-RM and FOD is required to get the best balance of low error rates (HTER) and calibration. Future work could explore refining this trade-off or applying this orthogonal decomposition strategy to other domain generalization tasks outside of face anti-spoofing.

Summary updated

Abstract

Domain Generalizable Face Anti-Spoofing (DGFAS) methods effectively capture domain-invariant features by aligning the directions (weights) of local decision boundaries across domains. However, the bias terms associated with these boundaries remain misaligned, leading to inconsistent classification thresholds and degraded performance on unseen target domains. To address this issue, we propose a novel DGFAS framework that jointly aligns weights and biases through Feature Orthogonal Decomposition (FOD) and Group-wise Scaling Risk Minimization (GS-RM). Specifically, GS-RM facilitates bias alignment by balancing group-wise losses across multiple domains. FOD employs the Gram-Schmidt orthogonalization process to decompose the feature space explicitly into domain-invariant and domain-specific subspaces. By enforcing orthogonality between domain-specific and domain-invariant features during training using domain labels, FOD ensures effective weight alignment across domains without negatively impacting bias alignment. Additionally, we introduce Expected Calibration Error (ECE) as a novel evaluation metric for quantitatively assessing the effectiveness of our method in aligning bias terms across domains. Extensive experiments on benchmark datasets demonstrate that our approach achieves state-of-the-art performance, consistently improving accuracy, reducing bias misalignment, and enhancing generalization stability on unseen target domains.
BibTeX
@inproceedings{jung2025group,
  author = {Seungjin Jung and Kanghee Lee and Yonghyun Jeong and Haeun Noh and Jungmin Lee and Jongwon Choi},
  title = {{Group-wise Scaling and Orthogonal Decomposition for Domain-Invariant Feature Extraction in Face Anti-Spoofing}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2025},
}
Figure