Palmprint and Face Multi-Modal Biometric Recognition Based on SDA-GSVD and Its Kernelization

Xiao-Yuan Jing, Sheng Li, Wen-Qian Li, Yong-Fang Yao, Chao Lan, Jiasen Lu, Jing-Yu Yang

TL;DR

This paper introduces a novel multimodal biometric recognition approach using Subclass Discriminant Analysis (SDA) to fuse palmprint and face data. By treating different biometric modalities of the same person as subclasses within a single class, the method effectively maximizes inter-person variability while minimizing intra-person variability. The authors also propose kernelized extensions (KPCA-SDA and KSDA-GSVD) to handle non-linear data distributions, achieving state-of-the-art performance.

What Problem Does This Paper Address?

Traditional single-modal biometric systems often struggle with issues like noisy data, restricted feature sets, and high error rates. Multimodal biometric systems aim to overcome these limitations by combining information from multiple sources (e.g., face and palmprint). However, a key challenge in multimodal recognition is effectively extracting discriminative features that capture the inherent variability within a person's different biometric traits while distinguishing them from others. Existing methods often overlook the underlying data distribution, treating all data from a single person as a single class. This paper addresses this by proposing a method that accounts for the fact that different biometric modalities from the same individual might exhibit different distributions (e.g., different Gaussian clusters) within the same feature space.

Methods

The core of the proposed method is Subclass Discriminant Analysis (SDA). Unlike traditional Linear Discriminant Analysis (LDA), SDA is designed to handle data where a single class can be represented by multiple subclasses, each with its own distribution. In this context, a person's face data and palmprint data are treated as two distinct subclasses belonging to the same "person" class. SDA then finds a transformation space that maximizes the separation between these subclasses of different individuals while minimizing the variation within each subclass.

To address the singularity problem that can arise in scatter matrix calculations, especially with limited training data, the authors propose two solutions:

  1. SDA-PCA: Principal Component Analysis (PCA) is first applied to reduce the dimensionality of each modality, followed by SDA for feature extraction.
  2. SDA-GSVD: Generalized Singular Value Decomposition (GSVD) is directly employed to solve the SDA problem, avoiding the singularity issue.

Furthermore, to capture non-linear relationships, the authors introduce kernelized versions:

  • KPCA-SDA: Kernel PCA (KPCA) is applied to each modality individually to map them into a high-dimensional feature space, followed by SDA for fusion.
  • KSDA-GSVD: Kernel SDA (KSDA) is directly applied to fuse multimodal data using GSVD, leveraging kernel functions to handle non-linearities.

Datasets

The proposed methods were evaluated on the following publicly available datasets:

  • AR face database: Contains over 4,000 color face images from 126 people.
  • FRGC database: Includes 12,776 training images from 222 individuals, with a focus on challenging uncontrolled conditions.
  • PolyU palmprint database: Comprises 3,780 images from 189 individuals.

Experiments involved fusing AR face data with PolyU palmprint data, and FRGC face data with PolyU palmprint data.

Key Insights and Ideas

  • Subclass Representation: The key insight is that different biometric modalities from the same individual can be viewed as distinct subclasses within a single person's class. This allows for a more nuanced representation than treating all data as a single monolithic class.
  • SDA's Suitability: SDA is a natural fit for multimodal biometrics because it can model the mixture-Gaussian distributions that often arise when combining different biometric traits from the same person.
  • Kernelization for Non-linearity: The success of KPCA-SDA and KSDA-GSVD highlights the importance of non-linear methods in capturing complex relationships within biometric data, especially when dealing with variations in pose, illumination, and expression.
  • GSVD for Robustness: Using GSVD as a solution to the singularity problem in SDA proves to be robust and effective, especially in scenarios with limited training samples.

Key Results

The proposed methods, particularly the kernelized versions, demonstrated superior performance compared to single-modal recognition and other multimodal fusion techniques.

  • Identification:
    • On the AR face and PolyU palmprint databases, SDA-GSVD achieved an average recognition rate of 98.23% ± 0.68%, outperforming other linear methods.
  • KPCA-SDA and KSDA-GSVD achieved average recognition rates of 98.74% ± 0.45% and 99.15% ± 0.63%, respectively, significantly outperforming single-modal KDA.
  • Similar trends were observed on the FRGC face and PolyU palmprint databases, with KSDA-GSVD achieving the highest average recognition rate.
  • Verification:
    • The proposed SDA-based approaches achieved significantly low Equal Error Rates (EERs), all below 1% on the AR/PolyU dataset, with KSDA-GSVD achieving the lowest EER of 0.56%.
  • On the FRGC/PolyU dataset, SDA-GSVD achieved an EER of 0.28%, demonstrating excellent verification performance.

Limitations and Future Work

  • Limitations: The paper primarily focuses on fusing two modalities (face and palmprint). While the authors suggest the approach can be extended to more modalities, this was not explicitly demonstrated. The computational cost of kernel methods, though not detailed, is a general consideration.
  • Future Work: The authors suggest that the proposed methods can be easily extended to multimodal biometric data recognition involving more than two modalities. Further exploration into optimizing the kernel parameters and exploring other kernel functions could also be beneficial.

Summary updated

Abstract

When extracting discriminative features from multimodal data, current methods rarely concern themselves with the data distribution. In this paper, we present an assumption that is consistent with the viewpoint of discrimination, that is, a person’s overall biometric data should be regarded as one class in the input space, and his different biometric data can form different Gaussians distributions, i.e., different subclasses. Hence, we propose a novel multimodal feature extraction and recognition approach based on subclass discriminant analysis (SDA). Specifically, one person’s different bio-data are treated as different subclasses of one class, and a transformed space is calculated, where the difference among subclasses belonging to different persons is maximized, and the difference within each subclass is minimized. Then, the obtained multimodal features are used for classification. Two solutions are presented to overcome the singularity problem encountered in calculation, which are using PCA preprocessing, and employing the generalized singular value decomposition (GSVD) technique, respectively. Further, we provide nonlinear extensions of SDA based multimodal feature extraction, that is, the feature fusion based on KPCA-SDA and KSDA-GSVD. In KPCA-SDA, we first apply Kernel PCA on each single modal before performing SDA. While in KSDA-GSVD, we directly perform Kernel SDA to fuse multimodal data by applying GSVD to avoid the singular problem. For simplicity two typical types of biometric data are considered in this paper, i.e., palmprint data and face data. Compared with several representative multimodal biometrics recognition methods, experimental results show that our approaches outperform related multimodal recognition methods and KSDA-GSVD achieves the best recognition performance.
BibTeX
@inproceedings{jing2012palmprint,
  author = {Xiao-Yuan Jing and Sheng Li and Wen-Qian Li and Yong-Fang Yao and Chao Lan and Jiasen Lu and Jing-Yu Yang},
  title = {{Palmprint and Face Multi-Modal Biometric Recognition Based on SDA-GSVD and Its Kernelization}},
  booktitle = {Sensors},
  year = {2012},
  url = {https://openreview.net/forum?id=BL4TPbTVfgO},
}

Loading related content...

Figure