Grounding-Aware Judge Reward Reinforcement Learning for Out-of-Domain Face Anti-Spoofing

Xiaoyong Yu, Xinge YuarXiv 2026 · arXiv

TL;DR

This paper introduces a grounding-aware judge reward to improve the out-of-domain (OOD) generalization of Multimodal Large Language Models (MLLMs) for face anti-spoofing. By using a 32B vision-language model as a "process supervisor" during reinforcement learning, the authors ensure that the model's reasoning is actually anchored in image evidence rather than relying on "hackable" shortcuts or hallucinations.

What Problem Does This Paper Address?

Face Anti-Spoofing (FAS)—distinguishing real faces from physical attacks like printed photos or video replays—is notoriously difficult to generalize. A model trained on one dataset (In-Domain) often fails miserably when deployed in a new environment (Out-of-Domain) because it latches onto spurious correlations, like specific lighting or background textures, rather than the actual physical artifacts of a spoof.

While MLLMs using Chain-of-Thought (CoT) reasoning are promising because they provide interpretable evidence, they suffer from a "verification gap." In math or coding, RL is easy because the answer is objectively right or wrong. In FAS, the reasoning chain (e.g., "I see a moiré pattern") has no rule-verifiable ground truth. If you only reward the final binary verdict (Real vs. Spoof), the model often learns to "hallucinate" evidence to justify a guess, leading to poor generalization.

Figure 1: Qualitative examples on genuine faces where CoT SFT falsely rejects the face (hallucinating glare/reflection as a spoof cue) while the grounding-aware RL model correctly grounds its verdict in the actual im- age content.

Methods

The authors propose a two-stage training pipeline:

  1. CoT Supervised Fine-Tuning (SFT): A Qwen2.5-VL-7B model is fine-tuned on high-quality, human-verified reasoning chains that describe physical artifacts.
  2. Grounding-Aware RL (GRPO): The model undergoes Group Relative Policy Optimization. Instead of just rewarding the correct final answer, they introduce a 32B vision-language judge to score the reasoning process.

The judge evaluates three specific dimensions on a 1–5 scale:

  • D1: Evidence Anchoring: Does the cited evidence (e.g., "screen frame") actually exist in the image?
  • D2: Logical Coherence: Does the path from observation to verdict make sense, or is there a logical leap?
  • D3: Domain Knowledge: Is the FAS terminology and judgment rule applied correctly?

Crucially, the judge is "anchored" to the ground-truth label. It doesn't ask "Is this a spoof?" but rather "Given that this is a spoof, does this reasoning honestly and professionally support that conclusion?"

Figure 2: Schematic of the overall reward design. We fine-tune an MLLM with CoT SFT, then apply GRPO with a grounding-aware judge reward that scores reasoning against the image, improving out-of-domain generalization.

Datasets

The models were trained and evaluated on three major physical attack benchmarks:

  • CelebA-Spoof: Used for In-Domain (ID) training and evaluation.
  • LCC-FASD: A large crowd-collected dataset used for OOD evaluation.
  • MSU-MFSD: A mobile-based dataset used for OOD evaluation, representing a significant domain shift.
  • MS-MFAD: The source of the 14K high-quality CoT annotations used for the initial SFT stage.

Key Insights and Ideas

  • The Image is the Label: Since there is no text-based ground truth for reasoning, the image itself must serve as the verification signal. A powerful enough vision-language model can act as a proxy for a human expert to check this grounding.
  • Verdict-Only Rewards are Hackable: Rewarding only the final answer encourages "false-rejection bias," where the model becomes overly sensitive to any noise, calling it a "spoof" just to be safe. This works ID but fails OOD.
  • Process vs. Outcome: The judge provides "process supervision." By penalizing hallucinations (D1) and logical gaps (D2), the RL stage forces the model to move away from memorized shortcuts and toward generalizable forensic reasoning.
  • Sequence-Level Importance Sampling: The authors found that standard token-level RL (like PPO) was too noisy for long reasoning chains. They adopted a sequence-level approach (GSPO) to stabilize training.

Key Results

The grounding-aware RL model achieved the best balance between ID accuracy and OOD generalization:

  • OOD Performance: On the LCC-FASD benchmark, the method reduced the Average Classification Error Rate (ACER) from 0.288 to 0.273. On the harder MSU-MFSD benchmark, it dropped from 0.431 to 0.409.
  • ID Preservation: The model maintained a very low ACER on CelebA-Spoof (0.035), significantly outperforming traditional baselines like ResNet-50 (0.134) and CLIP-ViT-B/16 (0.098).
  • Ablation Study: Removing the judge reward and using only "accuracy + format" rewards failed to improve performance on the harder MSU-MFSD benchmark, proving that the judge's grounding signal is the "active ingredient" for generalization.

Limitations and Future Work

  • Modest Gains: The authors acknowledge that the OOD improvements, while consistent, are limited. They attribute this to a "grounding-capability bottleneck" in the 32B judge itself.
  • Judge Reliability: The performance of the system is strictly bounded by how well the judge can perceive fine-grained artifacts. As vision-language models become more powerful (e.g., 70B+ models or specialized forensic models), the authors expect even larger gains.
  • Computational Cost: Training with a 32B judge in the loop requires significant GPU resources (the study used multiple NVIDIA H100s).

Summary updated

Abstract

Face Anti-Spoofing (FAS) must distinguish genuine faces from presentation attacks. A unified Multimodal Large Language Model (MLLM) is an attractive solution: it detects physical attacks (print, replay, mask) in one model and produces interpretable forensic evidence via Chain-of-Thought (CoT). A central obstacle to post-training such a model with reinforcement learning is that FAS reasoning has no rule-verifiable ground truth: only the binary verdict is checkable, while the reasoning chain cannot be verified by any hand-crafted rule. Optimizing the verdict alone is therefore a sparse, hackable signal. The model can reach the correct answer through a shortcut such as a false-rejection bias, which does not generalize. We address this with a grounding-aware judge reward. A 32B vision-language judge scores each reasoning chain against the image on evidence anchoring, logical coherence, and domain knowledge, turning the image itself into the verification signal. The reward is applied in a Group Relative Policy Optimization (GRPO) stage after CoT supervised fine-tuning. It is the judge, rather than the RL objective, that supplies the process supervision a verdict-only reward cannot. Our study starts from a model that already performs well In-Domain (ID) after CoT-based supervised fine-tuning and asks how to improve its Out-of-Domain (OOD) generalization. We find that grounding-aware rewards are an effective means to that end. We validate the method on OOD generalization, where the reward reduces ACER on two physical benchmarks (LCC-FASD 0.288 to 0.273, MSU-MFSD 0.431 to 0.409) while preserving ID accuracy (CelebA-Spoof 0.033 to 0.035). Against ResNet-50/101 and CLIP-ViT-B/16 baselines, the MLLM dominates ID and matches or surpasses the strongest baseline OOD. To our knowledge this is the first use of LLM-as-judge process supervision for FAS.
BibTeX
@misc{yu2026grounding,
  author = {Xiaoyong Yu and Xinge Yu},
  title = {{Grounding-Aware Judge Reward Reinforcement Learning for Out-of-Domain Face Anti-Spoofing}},
  year = {2026},
  eprint = {2611.09047},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09047},
}
Figure