LFLDNet: Lightweight Fingerprint Liveness Detection Based on ResNet and Transformer

Kang Zhang, Shu Huang, Eryun Liu, Heng Zhao2023 · DOI

TL;DR

This paper introduces LFLDNet, a novel lightweight deep learning architecture for fingerprint liveness detection that combines ResNet and Transformer components. By incorporating style transfer with CycleGAN, LFLDNet significantly improves its ability to detect fake fingerprints made from unknown materials, achieving high accuracy with a remarkably small model size.

What Problem Does This Paper Address?

Automatic fingerprint identification systems (AFIS) are widely used for authentication, but they are vulnerable to spoofing attacks where fake fingerprints are presented to fool the system. While hardware-based solutions exist, they are expensive and difficult to update. Software-based methods, which analyze fingerprint images, are more cost-effective and easier to maintain. However, existing software-based methods often struggle with generalization to "unknown" or novel spoof materials not seen during training. Furthermore, many deep learning approaches for this task can be computationally expensive and prone to overfitting, especially on smaller datasets. This paper aims to address these limitations by developing a lightweight, highly generalizable fingerprint liveness detection system.

Methods

The proposed method, LFLDNet, involves several key stages:

  • Foreground Segmentation and Patch Extraction: The fingerprint image is first processed to extract the foreground region, removing background noise. Then, the image is partitioned into local patches centered around the image's center. This approach avoids direct minutiae extraction, which can be unreliable for fake fingerprints, and creates an end-to-end trainable dataset.
  • Lightweight Network Architecture (LFLDNet): The core of the system is a novel network that combines a modified ResNet with a Transformer's Multi-Head Self-Attention (MHSA) mechanism. The MHSA module is integrated into the later stages of the ResNet architecture to allow the network to focus on global ridge structures and improve feature representation. The network is designed to be lightweight by reducing the number of convolutional kernels and depth.
  • Style Transfer with CycleGAN: To enhance generalization to unknown spoof materials, the authors employ CycleGAN. This technique transfers the "style" of known fake fingerprint materials to create synthetic fake fingerprints that mimic unknown materials. This effectively augments the training dataset with a wider variety of spoof textures, improving the model's robustness against novel attacks.

Datasets

The research utilizes publicly available datasets for fingerprint liveness detection:

  • LivDet 2011, LivDet 2013, and LivDet 2015: These are standard benchmarks for evaluating fingerprint liveness detection algorithms. The LivDet 2015 dataset is particularly noted for including fake fingerprints made from new and unknown materials in its test set, making it crucial for evaluating generalization capabilities.

Key Insights and Ideas

  • Patch-based processing without minutiae: Extracting local patches from the fingerprint center, rather than relying on potentially unreliable minutiae points from fake fingerprints, offers a more robust and computationally efficient approach.
  • Transformer for global context: Integrating the MHSA mechanism from Transformers into a CNN backbone allows the network to better capture global ridge patterns and structural information, which is vital for distinguishing live from fake fingerprints.
  • CycleGAN for generalization: Using CycleGAN to synthesize fake fingerprints from unknown materials is a powerful strategy to overcome the limitation of training data not covering all possible spoofing techniques. This significantly boosts the model's ability to handle novel attacks.
  • Lightweight design: Achieving high accuracy with a significantly reduced number of parameters (0.83M) is a key achievement, making the model more deployable on resource-constrained devices.

Key Results

  • Overall Accuracy: On the LivDet 2015 dataset, the proposed method achieved an average classification error of 1.72% across all sensors.
  • Small-Area Fingerprint Accuracy: The method demonstrated a high accuracy of 95.27% on small-area fingerprint recognition tasks.
  • Model Size: The LFLDNet model is remarkably lightweight, with only 0.83 million parameters, significantly smaller than comparable methods like Slim-ResCNN (2.15M) and MobileNet-V1/V2 (4.04M/2.19M).
  • Generalization Improvement: The style transfer using CycleGAN led to substantial improvements in the Ferrfakeunknown metric for various sensors on the LivDet 2015 dataset, indicating better performance against unknown spoof materials. For example, the Biometrika sensor saw an improvement from 2.60% to 1.20%.

Limitations and Future Work

  • Limitations: While the paper demonstrates strong performance, the authors acknowledge that the current method might still face challenges with extremely complex or novel spoofing techniques not covered by the training or synthesized data. The performance on the Digital Persona reader for small-area fingerprints was noted as poorer.
  • Future Work: The authors plan to explore more advanced fingerprint liveness detection technologies and further improve the accuracy. They also aim to investigate other GAN-based models to enhance cross-sensor generalization and better address fingerprint deception attacks.

Summary updated

BibTeX
@misc{zhang2023lfldnet,
  author = {Kang Zhang and Shu Huang and Eryun Liu and Heng Zhao},
  title = {{LFLDNet: Lightweight Fingerprint Liveness Detection Based on ResNet and Transformer}},
  year = {2023},
  doi = {10.3390/s23156854},
}
Figure