TL;DR
This paper introduces C2CL, an end-to-end system for matching contactless fingerprint images (like those from smartphone photos) to traditional contact-based fingerprint databases. It achieves state-of-the-art performance by addressing key challenges like low contrast and varying image quality through a novel preprocessing pipeline and a hybrid minutiae-texture matching approach.
What Problem Does This Paper Address?
Fingerprint recognition is a cornerstone of biometric security, but a significant challenge arises from the increasing prevalence of contactless fingerprint capture (e.g., smartphone photos) versus the vast existing databases of contact-based fingerprints. Matching these two types of impressions is difficult due to several factors: contactless images often suffer from low ridge-valley contrast, non-uniform illumination, and significant variations in finger pose (roll, pitch, yaw) and distance to the camera. Conversely, contact-based fingerprints can exhibit non-linear distortions due to skin elasticity. Bridging this "domain gap" to achieve accurate matching is crucial for leveraging existing fingerprint infrastructure with new, more hygienic contactless capture methods, especially in light of public health concerns like the COVID-19 pandemic.
Methods
C2CL tackles this problem with a comprehensive, end-to-end system comprising several key modules:
- Preprocessing Pipeline: This is the core innovation for bridging the domain gap. It includes:
- Segmentation: A U-net based segmentation network robustly extracts the distal phalange (fingertip) of the contactless fingerprint, removing background noise.
- Enhancement: Adaptive histogram equalization and pixel gray-level inversion are used to improve ridge-valley contrast and correct for the inverted ridge/valley appearance common in contactless images.
- Scaling: A spatial transformer network (STN) is trained to normalize the resolution of contactless fingerprints to match the standard 500 pixels per inch (ppi) of contact-based impressions.
- Distortion Correction: The same STN also learns to correct for non-linear distortions (perspective and elastic deformation) present in contactless images, aiming to align them with the appearance of contact-based prints. This correction is supervised by the match scores of a pre-trained DeepPrint network, optimizing the warping to maximize genuine match scores.
- Representation Extraction and Matching: C2CL employs a hybrid approach:
- Texture Representation: A fine-tuned DeepPrint network (a deep learning model known for robust fingerprint representation) is used to extract a deep textural feature representation.
- Minutiae Representation: Traditional minutiae features (ridge endings and bifurcations) are extracted from the preprocessed contactless fingerprints using the Verifinger 12.0 SDK.
- Score Fusion: The final match score is a weighted fusion of the scores obtained from matching the texture representations and the minutiae representations.
Datasets
The authors utilized a combination of publicly available datasets and a newly collected dataset for their evaluations:
- Publicly Available Datasets:
- PolyU Contactless 2D to Contact-based 2D Images Database
- UWA Benchmark 3D Fingerprint Database
- ISPFDv2 Dataset
- MSU Finger Photo and Slap Fingerprint Database
- IIT Bombay Touchless and Touch-Based Fingerprint Database
- Newly Collected Dataset:
- ZJU Finger Photo and Touch-based Fingerprint Database: This dataset, collected in coordination with Zhejiang University, comprises 9,888 contactless and 9,888 contact-based fingerprint images from 206 subjects. It was captured using commodity smartphones and a URU 4500 optical scanner. This dataset is made publicly available to foster further research.
Key Insights and Ideas
- End-to-End Solution: The paper presents a truly end-to-end system, addressing all stages from capture (via a custom app) to matching, which is a significant advancement over prior work focusing on isolated sub-problems.
- Hybrid Representation is Key: Relying solely on minutiae for contactless prints is problematic due to low contrast and spurious features. The fusion of robust texture features (from DeepPrint) with minutiae significantly improves accuracy, especially in challenging scenarios.
- Learned Distortion Correction with DeepPrint Supervision: Using DeepPrint's invariance to certain variations as a supervisory signal for learning distortion correction is a clever way to guide the STN towards producing representations that are more compatible with contact-based prints, overcoming limitations of direct image comparison.
- Generalizability is Crucial: The authors emphasize the importance of cross-database evaluation (testing on datasets not seen during training) as a more realistic measure of performance in real-world scenarios, demonstrating C2CL's robustness.
- Smartphone Capture is Viable: The success with images captured by commodity smartphones highlights the potential for widespread adoption of contactless fingerprint recognition without specialized hardware.
Key Results
- State-of-the-Art Performance: C2CL achieves Tar@FAR=0.01% in the range of 96.67% to 98.30% across various datasets, outperforming previous SOTA methods.
- Competitive Accuracy: Even on the challenging ZJU dataset (a cross-database evaluation), C2CL achieves competitive performance with contact-contact matching, with an EER of 0.62% and TAR@FAR=0.01% of 97.56%.
- Significant Improvement from Components: Ablation studies show substantial gains from each preprocessing step, with enhancement and scaling providing particularly large improvements.
- Efficient Segmentation: The segmentation network achieves high Intersection Over Union (IOU) scores and is remarkably fast (12.6ms per image), crucial for real-time applications.
- Effective Search: The proposed two-stage search algorithm (DeepPrint for initial retrieval, Verifinger for re-ranking) achieves high accuracy on a large gallery (1.1 million fingerprints) with significant time savings compared to using Verifinger alone.
Limitations and Future Work
- Challenging Scenarios: While robust, the segmentation network can still fail in extremely challenging background and illumination settings.
- Computational Cost: While efficient, the system still requires significant computational resources, and further optimization is needed for deployment on resource-constrained mobile devices.
- Minutiae Extraction: The reliance on a COTS SDK for minutiae extraction and matching is a potential bottleneck for speed. Future work could involve using DeepPrint's minutiae output directly with a computationally efficient minutiae matcher.
- 3D Fingerprint Modeling: Acquiring multiple image views to build a 3D model of the finger is suggested as a future avenue, though it would increase computational cost and user burden.
- Data Augmentation and Pretraining: The authors note that further data augmentation and leveraging more publicly available datasets could improve performance, especially when pretraining DeepPrint on smaller datasets.
Summary updated