AVN: An Adversarial Variation Network Model for Handwritten Signature Verification

Huan Li, Ping Wei, Ping Hu

Abstract: Handwritten signature verification is a critical task in security and forensics, yet it is often hampered by the sparsity of stroke information and the sensitivity of deep learning models to superficial image variations. This paper introduces the Adversarial Variation Network (AVN), a novel framework designed to enforce "variation consistency." By utilizing an adversarial "variator" module that generates challenging pixel-intensity variants, the system learns to ignore non-essential image characteristics and focus on the underlying geometric structures of signature strokes. Experimental results on four diverse datasets—CEDAR, BHSig-Bengali, BHSig-Hindi, and GPDS—demonstrate that AVN outperforms existing state-of-the-art methods and exhibits superior robustness to environmental disturbances.

Introduction: The Challenge of Variation Consistency

Signature verification suffers from two primary hurdles: information sparsity and style vulnerability. Because signature strokes are thin, they occupy a minimal fraction of an image's pixels. Standard deep learning models often "cheat" by relying on image intensities or background noise rather than the actual geometry of the strokes.

As illustrated in Figure 1, a state-of-the-art model may correctly identify a signature as genuine with high confidence, yet fail completely when the image intensity is slightly modified (e.g., multiplied by 0.85). This inconsistency indicates that the model has not truly mastered the invariant "style" of the signature. The AVN model addresses this by forcing the network to be invariant to these superficial variations, a property termed variation consistency.

Demonstration of the failure of variation consistency in standard models.

Methodology: The Adversarial Variation Network

The AVN consists of three modules trained in an end-to-end min-max game:

  1. The Extractor ($F$): A VGG-inspired CNN that extracts deep discriminative features from reference and test signatures.
  2. The Discriminator ($D$): A module that classifies the extracted features as genuine or forged.
  3. The Variator ($V$): An adversarial module that generates a variation map—a weight mask applied to the original images to create variants with different intensities.

The training process involves a "cooperation-competition" loop. The Variator strives to maximize the loss by creating complex, challenging variants (as shown in Figure 3) that might confuse the verifier. Simultaneously, the Extractor and Discriminator cooperate to minimize the loss, learning to make correct decisions despite these adversarial perturbations. This forces the Extractor to ignore pixel intensity and focus strictly on invariant stroke structures.

The Variator generates a sequence of challenging variants to train the model for robustness.

Experimental Results

The model was evaluated on four multi-language datasets: CEDAR (English), BHSig-Bengali, BHSig-Hindi, and the GPDS Synthetic Signature dataset. Key results include:

  • CEDAR: Achieved 96.16% accuracy, surpassing previous SOTA methods like HOCCNN (95.06%).
  • BHSig-Bengali: Reached 93.80% accuracy, a significant improvement over the previous best of 88.08%.
  • GPDS: Achieved 90.32% accuracy and a low 9.77% Equal Error Rate (EER) on this massive dataset.

Robustness testing confirmed the efficacy of the adversarial training. While baseline models saw their accuracy plummet under heavy intensity variation (e.g., dropping from 90.74% to 50.00% on BHSig-Hindi), the AVN remained stable, maintaining 89.93% accuracy.

Conclusion

The AVN model demonstrates that active adversarial mining of features significantly improves the robustness and accuracy of signature verification systems. By shifting the focus from "ink" to the "hand" through variation consistency, the model achieves superior generalization. Future work will explore replacing the simple convolutional backbone with more sophisticated architectures like ResNet and extending the framework to other biometric tasks such as fingerprint or iris recognition.

Summary updated

Abstract

Handwritten signature verification is a crucial yet challenging problem. While previous studies have made great progress in this problem, they learn signature features passively from given existing data. In this paper, we propose a novel adversarial variation network (AVN) model for handwritten signature verification which mines effective features by actively varying existing data and generating new data. Powered by a proposed novel variation consistency mechanism, the AVN contains three different types of modules unified under one end-to-end framework: the extractor seeks to extract deep discriminative features of handwritten signatures, the discriminator aims to make verification decisions based on the extracted features, and the variator is designed to actively generate signature variants for constructing a more discriminative model. The proposed model is trained in an adversarial way with a min-max loss function, by which the three modules cooperate and compete to enhance the entire model’s ability and therefore the signature verification performance is improved. We test the proposed method on four challenging signature datasets of different languages: CEDAR, BHSig-Hindi, BHSig-Bengali, and GPDS Synthetic Signature. Extensive experiments with in-depth discussions validate the effectiveness of the proposed method.
BibTeX
@article{li2022avn,
  author = {Huan Li and Ping Wei and Ping Hu},
  title = {{AVN: An Adversarial Variation Network Model for Handwritten Signature Verification}},
  journal = {IEEE Trans. Multimedia},
  year = {2022},
  doi = {10.1109/tmm.2021.3056217},
}

Loading related content...

Figure