A White-Box False Positive Adversarial Attack Method on Contrastive Loss-Based Offline Handwritten Signature Verification Models

Zhongliang Guo, Yifei Qian, Ognjen Arandjelovic, Lei Fang

Abstract

Modern offline handwritten signature verification systems increasingly rely on Siamese neural networks trained with contrastive loss to authenticate signatures. While robust against random variations, these models exhibit a critical vulnerability to targeted adversarial attacks. This paper presents a novel white-box attack framework that treats adversarial perturbation as a style transfer problem. By optimizing a forged signature to adopt the stroke characteristics and texture of a genuine signature, the proposed method achieves near-perfect success rates in generating false positives, significantly outperforming traditional gradient-based adversarial attacks.

Introduction

Handwritten signature verification is a vital biometric tool for legal and financial authentication. Current state-of-the-art models utilize deep learning to map signature images into a high-dimensional latent space, where the distance between a genuine signature and a forgery is maximized. However, these models are susceptible to adversarial attacks—small, intentional perturbations designed to mislead the AI.

Traditional adversarial methods like the Fast Gradient Sign Method (FGSM) or Projected Gradient Descent (PGD) were primarily designed for object recognition tasks. When applied to signatures, these methods often fail because signature images are sparse, consisting of thin ink strokes on a clean white background. Standard noise-based perturbations either become visually obvious or fail to bridge the gap in the model's latent space, resulting in a 0% success rate for False Positive (FP) attacks—where a forgery is incorrectly accepted as genuine.

Methodology

The core innovation of this research is the conceptualization of an adversarial attack as a style transfer task. In a white-box setting, where the attacker has full access to the target model's architecture and weights, the forged image itself is treated as a set of trainable parameters.

Framework of the proposed white-box style-transfer attack.

As illustrated in the framework above, the method utilizes a composite loss function to iteratively update a synthesized image until it successfully deceives the Siamese network. The loss function consists of four components:

  1. Attack Loss: Minimizes the distance in the embedding space between the synthesized image and the target genuine signature.
  2. Style Loss: Employs Gram matrices to align the texture and stroke habits of the forgery with those of the genuine signer.
  3. Difference Loss: Ensures the synthesized image remains visually similar to the original forgery at the pixel level to maintain stealth.
  4. Total Variation (TV) Loss: Acts as a regularizer to prevent high-frequency artifacts and ensure visual smoothness.

Experimental Results

The proposed method was evaluated against two benchmark datasets: CEDAR (1,320 genuine and 1,320 forged signatures) and BHSig260-B (Bengali script signatures from 160 writers).

Performance comparison of various attack methods on CEDAR and BHSig260-B datasets.

As shown in the table above, the style-transfer approach dramatically outperforms existing techniques. On the CEDAR dataset, the attack achieved a 99.71% success rate for False Positives, while standard PGD and FGSM methods failed completely (0% success). On the more linguistically diverse BHSig260-B dataset, the method maintained a high success rate of 82.93%.

Furthermore, the attack demonstrated significant resilience to common defenses. When the target model was protected using Adversarial Training (AT), the success rate on CEDAR remained as high as 93.7%, indicating that the style-based perturbations are fundamentally different from the noise patterns that traditional adversarial training aims to mitigate.

Conclusion

This research exposes a significant security flaw in contrastive loss-based signature verification models. By leveraging style transfer principles, attackers can generate forgeries that are mathematically indistinguishable from genuine signatures to a Siamese network while remaining visually plausible to human observers. Future work will focus on extending these principles to black-box scenarios where model internals are unknown, and investigating more robust defensive architectures to counter style-based adversarial perturbations.

Summary updated

BibTeX
@inproceedings{guo2024white,
  author = {Zhongliang Guo and Yifei Qian and Ognjen Arandjelovic and Lei Fang},
  title = {{A White-Box False Positive Adversarial Attack Method on Contrastive Loss-Based Offline Handwritten Signature Verification Models}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2024},
}
Figure