Validation-Guided Prompt Selection for Differential Morphing Attack Detection

Raghavendra Ramachandra, Hailin LiarXiv 2026 · arXiv

TL;DR

Researchers have demonstrated that for face morphing attack detection, how you ask a Vision-Language Model (VLM) matters as much as the model itself. By using a small validation set to automatically select the best prompt from a pool of candidates, they reduced the missed-attack rate of a frozen GPT-4o model from 70% to under 20% without any fine-tuning.

What Problem Does This Paper Address?

Face morphing attacks are a serious threat to automated border control. An attacker blends photos of two different people into a single image that can successfully be verified against both identities. Detecting these is hard because new morphing techniques (using GANs or Diffusion models) emerge constantly, and high-quality biometric training data is scarce.

The authors focus on Differential Morphing Attack Detection (D-MAD), where a system compares a "trusted" live capture (like at a gate) against a document image (the passport). While Vision-Language Models (VLMs) like GPT-4o are great at comparing images "out of the box," their performance is highly sensitive to the specific instructions (prompts) they are given. This paper investigates whether we can treat prompt design as a supervised selection problem—using a tiny bit of labeled data to find the "perfect" instruction rather than trying to retrain the massive model itself.

Methods

The authors propose an agentic framework to find the best D-MAD prompt, keeping the underlying VLM (GPT-4o) completely frozen.

Figure 1. Agentic prompt generation framework for D-MAD. Version 1 and Version 2 Generation 0 follow the solid path from prompt generation to validation. Recursive Version 2 candidates follow the dashed path from the Scoring Module through the Prompt Optimizer and Prompt Critic before returning to the Candidate Pool. The selected prompt is fixed before test evaluation. Test responses are subsequently combined by deterministic decision and score fusion.

They tested two versions:

  1. Version 1 (Non-Recursive Selection): A "Prompt Generator Agent" creates 50 candidate prompts. These are ranked on a small validation set using a custom Security Loss ($\mathcal{L}_{sec}$). This loss function is heavily weighted toward reducing missed attacks (MACER) but also penalizes false alarms (BSCER), "I don't know" responses (FTA), and inconsistent decisions across multiple tries.
  2. Version 2 (Recursive Optimization): This version uses a "Prompt Optimizer Agent" and a "Prompt Critic Agent." It starts with 30 prompts and iteratively refines them based on aggregate feedback from the validation set (e.g., "this prompt is missing too many GAN-based morphs").

To improve reliability, the system queries the model three times per image pair and uses an OR-fusion rule: if any of the three responses flags a morph, the system treats it as an attack.

Datasets

The study uses a D-MAD dataset consisting of 50 subjects.

  • Validation Set: 10 subjects (used only for prompt selection/optimization).
  • Test Set: 40 subjects (subject-disjoint, meaning the model never saw these people during the prompt selection phase).
  • Morphing Techniques: The images include landmark-based blending (LMA), GAN-based synthesis (MIPGAN-2), and diffusion-based generation (PIPE).

Key Insights and Ideas

  • Prompting is a "First-Class" Variable: The difference between a "reasonable" manual prompt and the best-selected prompt was a staggering 50 percentage points in error rate. This suggests that in VLM-based security tasks, prompt engineering shouldn't be an afterthought—it's a core part of the algorithm.
  • The "Over-Optimization" Trap: Surprisingly, the recursive optimization (Version 2) performed worse on the test set than the simple selection (Version 1). With only 10 subjects in the validation set, the recursive agent likely "overfit" the prompts to the specific quirks of those 10 people, failing to generalize as well as the broader pool of initial candidates.
  • LMA is the "Final Boss": While the VLM became nearly perfect at detecting GAN and Diffusion morphs, landmark-based morphs (LMA) remained difficult. This indicates that VLMs might be overly reliant on "texture" artifacts (which GANs/Diffusion leave behind) and struggle with the subtle geometric inconsistencies of traditional blending.
  • Reliability is a Metric: The authors explicitly included "parse failures" and "inconsistency" in their loss function. In a real-world border control setting, a model that gives a "correct" answer but crashes 5% of the time is a failure.

Key Results

The quantitative jump in performance was significant, as shown in the comparison between the manual baseline and the automated selection (V1: P033).

Table 1. Overall test performance on 40 bona fide and 120 attack pairs. Version 1 and Version 2 prompts are selected using labelled validation pairs; no model parameters are updated. Lower values are better.
  • MACER (Missed Attacks): Dropped from 70.00% (Manual) to 19.17% (V1).
  • BSCER (False Alarms): Maintained at 0.00% for the top-performing prompt.
  • Morph-Specific Success: As seen in the table below, the V1 prompt achieved 0.00% error on both MIPGAN-2 and PIPE morphs, though it still missed 57.5% of LMA morphs.
Table 2. Test MACER by morphing method, with missed attacks among 40 pairs in parentheses. Lower is better.

Limitations and Future Work

  • Validation Bottleneck: The small validation set (10 subjects) limited the effectiveness of recursive optimization. Future work should explore the "sweet spot" for validation size to enable true prompt evolution without overfitting.
  • Black-Box Dependency: The study relies on the gpt-4o API. Because these models are updated by providers behind the scenes, the "best prompt" today might not be the best prompt tomorrow.
  • LMA Gap: The continued difficulty with LMA morphs suggests that future prompts need to steer the VLM more toward "identity-level" comparisons rather than just looking for surface-level blending artifacts.
  • Privacy: Using cloud-based VLMs for biometric data raises significant privacy and consent issues, which the authors acknowledge as a major constraint for deployment.

Summary updated

Abstract

Differential morphing attack detection (D-MAD) compares a trusted live capture with a document image that may contain a facial morph. We investigate whether labelled validation data can improve a frozen-parameter vision-language evaluator through prompt selection rather than model fine-tuning. Version~1 generates 50 candidate prompts and selects the top-ranked instruction using a security-oriented loss that prioritises missed morph attacks (MACER) while accounting for bona fide classification errors (BSCER), response failures, parse failures, and decision inconsistency. Version~2 recursively revises prompts using aggregate validation feedback. On a subject-disjoint test set of 160 pairs spanning LMA, MIPGAN-2, and PIPE, the selected Version~1 prompt achieves a (19.17\%) morph attack classification error rate (MACER), with (0.00\%) observed bona fide sample classification error rate (BSCER), compared with (70.00\%) MACER for a controlled manual prompt. The paired (50.83) percentage-point difference has a (95\%) subject-bootstrap interval of ([43.33,58.33]). Recursive revision does not surpass the best Generation~0 validation loss; the reported revised descendant obtains (31.67\%) test MACER. All 23 attacks missed by Version~1 are landmark-based (LMA) morphs. Overall, the results show that prompt formulation can substantially influence frozen-parameter VLM-based D-MAD performance, while also highlighting the challenge of recursive prompt refinement under a compact 10-subject validation setting.
BibTeX
@misc{ramachandra2026validation,
  author = {Raghavendra Ramachandra and Hailin Li},
  title = {{Validation-Guided Prompt Selection for Differential Morphing Attack Detection}},
  year = {2026},
  eprint = {2611.17176},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17176},
}
Figure