TL;DR
This paper introduces a novel "motion transfer" technique to generate realistic finger vein images with varying poses. By modeling actual finger movements, their method significantly improves the accuracy of finger vein recognition systems, especially when combined with synthetic data generation.
What Problem Does This Paper Address?
Finger vein recognition (FVR) is a secure biometric method, but its performance is often limited by the scarcity of public vein datasets. Traditional data augmentation techniques, like rotation or cropping, fail to capture the natural variations in finger posture and movement, leading to limited improvements. While deep learning methods have shown promise, they too struggle with generating realistic vein images that accurately reflect complex, real-world finger motions, hindering the development of robust FVR systems.
Methods
The core of this paper is a "motion transfer" (MT) model designed for intra-class data augmentation. The process involves:
- Key Point and Pose Detection: A UNet-based network extracts key points and estimates the pose (orientation and confidence) of the finger from both a "source" image and a "driving" image (which dictates the target pose). This is treated as a unified entity, not articulated parts.
- Dense Motion Flow Estimation: The differences in key points and pose heatmaps between the source and driving images are used to estimate a sparse motion flow. This sparse flow is then used to generate a coarse dense motion flow. A separate network refines this into a final dense motion flow, which guides the warping process. A mask is also learned to identify areas needing inpainting.
- Warping-based Image Generation: Using a UNet architecture similar to FOMM, the model warps the feature embedding of the source image based on the estimated dense motion flow. An inpainting mask is used to fill in any newly exposed regions, generating a new image with the target pose.
- Training Loss: The model is trained using a combination of perceptual loss (comparing generated and target images using VGG19 features) and an equivariance loss for self-supervised key point detection.
The authors then integrate this MT model into an "MT-Aug" pipeline for FVR training. This involves analyzing principal motion vectors from intra-class samples to learn generic motion patterns, which are then used to generate novel poses for existing or synthetic finger vein images.
Datasets
The proposed method was evaluated on three publicly available finger vein databases:
- FV-USM: 5904 images from 492 classes.
- MMCBNU: 6000 images from 600 classes.
- SDUMLA: 3816 images from 628 classes.
For training the FVR baseline, they also utilized synthetic finger vein samples generated by StyleGAN2, treating each synthetic sample as a distinct inter-class sample. An in-house dataset of 252 videos (each with 120 frames) was used for qualitative visualization of motion transfer.
Key Insights and Ideas
- Modeling Real Motion is Crucial: The paper strongly emphasizes that accurately capturing the complex, non-rigid motion of a finger is key to effective data augmentation for FVR, a limitation of previous methods.
- Motion Transfer for Intra-Class Augmentation: Applying motion transfer specifically for intra-class augmentation (generating variations of the same person's finger) is shown to be highly effective, going beyond simple geometric transformations.
- Synergy with Synthetic Data: Combining their MT-Aug with synthetic data generation (from StyleGAN2) yields even better results than using synthetic data alone, suggesting a powerful complementary approach.
- Lightweight and Efficient: The MT model is described as lightweight and capable of online augmentation, making it practical for real-world FVR systems.
Key Results
- The proposed MT-Aug method significantly improves recognition accuracy across all three public datasets compared to baseline methods (e.g., WLD, RLT, MC, and FusionAug).
- When combined with synthetic samples, MT-Aug achieved substantial improvements, particularly for the challenging SDUMLA database, outperforming baseline methods and even synthetic data augmentation without MT.
- The MT model is efficient, with a parameter count of 233.7k and GFLOPs of 0.1246, achieving inference times of 10.68 ms on a high-end CPU.
- Qualitative results demonstrate the model's ability to generate realistic finger rolling, horizontal, and vertical translations.
Limitations and Future Work
- Not discussed in the paper. The authors focus on presenting their successful method and its validation, without explicitly detailing limitations or future work directions in the provided text.
Summary updated