Synthetic Latent Fingerprint Generator

André Brasil Vieira Wyzykowski, Anil K. JainWACV 2023

TL;DR

This paper introduces a novel CycleGAN-based approach to generate synthetic latent fingerprints with controllable quality (Good, Bad, Ugly). This method addresses the scarcity of public latent fingerprint datasets, enabling better training of fingerprint recognition systems and improving their performance, particularly on challenging latent prints.

What Problem Does This Paper Address?

Fingerprints are crucial evidence in criminal investigations, but matching latent prints (found at crime scenes) to rolled/slap prints in databases is a significant challenge. Latent prints are often of poor quality due to noise, blurriness, and partial impressions. Research in this area is hampered by the lack of large, diverse, and publicly available latent fingerprint datasets, especially those captured from real crime scenes. Existing synthetic fingerprint generators typically focus on generating high-quality rolled or plain prints and don't address the specific challenges of latent prints. This scarcity limits the development and evaluation of robust latent fingerprint recognition systems.

Methods

The authors propose a two-stage CycleGAN-based approach to generate synthetic latent fingerprints.

  • Stage 1: Coarse Latent Generation: A global CycleGAN model is trained to translate rolled fingerprints into a generic latent domain. This stage uses a modified CycleGAN architecture with both global and patch discriminators to improve detail generation and stabilize training.
  • Stage 2: Style-Controlled Latent Generation: This stage refines the latent generation process by clustering latent fingerprints based on visual similarity.
  • Clustering: A pre-trained ResNet152V2 model extracts feature vectors from latent images, which are then clustered using K-Means. The authors specifically use three clusters, mirroring the "Good," "Bad," and "Ugly" categories found in the NIST SD27 database.
  • Finetuning: For each cluster, a specific CycleGAN model is finetuned. This allows for the generation of latent fingerprints with distinct styles, mimicking the variations seen in real-world latents (e.g., different levels of distortion, texture, and background). Data augmentation (random translations and rotations) is applied during training to improve convergence.

The overall goal is to generate synthetic latent prints that are visually similar to real latents and can be controlled in terms of their quality and difficulty for recognition.

Datasets

  • Training Data:
    • NIST SD4 (rolled prints)
    • MSP latent database (used for texture transfer and clustering)
  • Evaluation Data:
    • NIST SD27 (latent prints)
    • NIST SD302 (latent prints)
    • IIITD-SLF (latent prints)
    • MSP latent database

The paper leverages existing public datasets but aims to create a new synthetic dataset that can be publicly shared to aid research.

Key Insights and Ideas

  • Controllable Latent Quality: The ability to generate "Good," "Bad," and "Ugly" latent fingerprints is crucial. This allows researchers to specifically target and improve performance on the most challenging types of latents, which are often the bottleneck in real-world systems.
  • CycleGAN for Latent Synthesis: Adapting CycleGAN, which is typically used for image-to-image translation between domains, for the specific task of generating latent fingerprints from rolled prints is a key innovation. The addition of patch discriminators is vital for capturing fine details.
  • Clustering for Style Diversity: Clustering latent fingerprints based on visual similarity before finetuning CycleGAN models is essential for capturing the diverse styles and quality variations present in real latent impressions.
  • Synthetic Data for Augmentation: The synthetic latent prints can effectively augment existing small latent training databases, leading to significant improvements in the performance of state-of-the-art latent fingerprint matchers.

Key Results

  • Qualitative Similarity: Visual comparisons (Figure 5) and t-SNE embeddings (Figure 7) show that the generated synthetic latents are visually similar to real latents from NIST SD27 and MSP databases.
  • Quality Overlap: NFIQ 2 quality scores for synthetic latents overlap with those of real latents from NIST SD27 and MSP databases (Figure 6).
  • Recognition Difficulty: ROC curves for synthetic latents (Good, Bad, Ugly) show similar recognition difficulty trends compared to their real counterparts in NIST SD27 (Figure 9).
  • Performance Improvement: Finetuning the DeepPrint matcher with synthetic latent data (specifically "Bad" and "Ugly" latents) led to significant improvements in Rank-1 accuracy on NIST SD27, NIST SD302, and IIITD-SLF. For instance, Rank-1 accuracy on NIST SD27 improved from 15.50% to 29.07% with a fused model (DeepPrint4).
  • Minutiae Count: The number of minutiae in generated latents generally decreases with lower quality categories (Good, Bad, Ugly), mirroring real latents. The mean minutiae counts for synthetic latents were comparable to those in NIST SD27.

Limitations and Future Work

  • Limitations:
    • The quality of synthetic latents is similar to MSP latents, which have a different quality distribution than NIST SD27. The goal was not to replicate NIST SD27 but to generate diverse quality latents for augmentation.
  • The paper acknowledges that the alignment of rolled and latent mates is still a challenging task when using real latent data for training, a problem that is avoided with synthetic data.
  • Future Work:
    • Refine the synthesis of latent prints by adding more complexity to background and texture variations.
  • Explore the capability of CycleGAN to revert a latent into a rolled one, which could further improve latent recognition in deep networks.

Summary updated

BibTeX
@inproceedings{wyzykowski2023synthetic,
  author = {André Brasil Vieira Wyzykowski and Anil K. Jain},
  title = {{Synthetic Latent Fingerprint Generator}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure