TL;DR
This paper introduces a novel "complete" Kernel Fisher Discriminant Analysis (CKFD) algorithm that improves upon existing methods by leveraging both within-class and null-space discriminant information for face recognition. By decomposing the process into Kernel Principal Component Analysis (KPCA) followed by Fisher Linear Discriminant Analysis (FLD), CKFD offers a more transparent and powerful approach, achieving significantly better recognition rates on the FERET database.
What Problem Does This Paper Address?
Face recognition, a challenging task in computer vision, often relies on feature extraction techniques to reduce dimensionality and highlight discriminative information. Principal Component Analysis (PCA) and Fisher Linear Discriminant Analysis (FLD) are classic linear methods. However, for complex, non-linear data like faces, non-linear extensions are needed. Kernel PCA (KPCA) and Kernel FLD (KFD) address this by implicitly mapping data to a high-dimensional feature space.
A key challenge with existing KFD algorithms is their tendency to ignore discriminant information residing in the "null space" of the within-class scatter matrix. This null space, while seemingly problematic, actually contains valuable information, especially for tasks like face recognition where subtle variations are crucial. Existing KFD methods, by focusing only on information outside this null space, are thus limited in their discriminatory power. This paper aims to develop a KFD algorithm that can effectively utilize all available discriminant information, both within and outside the null space.
Methods
The core contribution is the development of a "complete" Kernel Fisher Discriminant Analysis (CKFD) algorithm. The authors propose a two-phase approach:
- Kernel Principal Component Analysis (KPCA): This phase maps the original input data into a high-dimensional feature space and performs PCA in this space. This is achieved efficiently using kernel tricks, avoiding explicit computation in the high-dimensional space. This step is equivalent to performing PCA in the input space.
- Fisher Linear Discriminant Analysis (FLD): Following KPCA, FLD is applied in the transformed space. The novelty here lies in how FLD is implemented. Instead of discarding the null space of the within-class scatter matrix, CKFD splits the problem into two categories of discriminant features:
- Category I: Discriminant features extracted using the classical Fisher criterion, focusing on information outside the null space.
- Category II: Discriminant features extracted by leveraging the information within the null space.
This two-phase decomposition (KPCA followed by FLD) is presented as a more transparent and simpler framework. The algorithm uses a polynomial kernel, $k(x, y) = (x \cdot y + 1)^r$, with $r=2$ for computational efficiency.
Datasets
The proposed algorithm was evaluated on a subset of the FERET face image database. This subset comprised:
- 1400 images from 200 individuals.
- Each individual had 7 images.
- Images were named with specific two-character strings indicating variations in facial expression, illumination, and pose (up to $\pm 15^\circ$ and $\pm 25^\circ$).
- The experimental setup involved cropping the facial portion, resizing to 80x80 pixels, and applying histogram equalization.
- For training, 3 images per subject were randomly chosen (total 600 training samples).
- The remaining 4 images per subject were used for testing (total 800 testing samples).
Key Insights and Ideas
- Leveraging the Null Space: The most significant insight is that the null space of the within-class scatter matrix, often discarded, contains crucial discriminant information for face recognition. CKFD's ability to exploit this information is its primary advantage.
- Two-Phase Decomposition: Breaking down KFD into KPCA followed by FLD makes the algorithm more transparent and easier to understand and implement.
- Enhanced Discriminatory Power: By combining discriminant information from two distinct categories (outside and within the null space), CKFD can capture more subtle facial variations, leading to improved recognition performance.
- Generalization of FLD: The paper implicitly shows that FLD can be effectively performed in a PCA-transformed space, a concept that is further extended by KPCA.
Key Results
The experimental results, summarized in Table 1, demonstrate the superiority of CKFD:
- CKFD (214 features), which combines 199 features from Category I and 15 from Category II, achieved an average correct recognition rate of 81.97%.
- This significantly outperformed Kernel Eigenface (average 29.43%) and Kernel Fisherface (average 73.43%).
- CKFD using only Category I features (199 features) achieved an average recognition rate of 73.40%, which was comparable to Kernel Fisherface. This highlights that Kernel Fisherface essentially only utilizes the information outside the null space.
- The improvement observed when incorporating Category II features for CKFD underscores the importance of the null-space information.
Limitations and Future Work
- Computational Cost: While the kernel trick avoids explicit high-dimensional computation, kernel methods can still be computationally intensive, especially with large datasets or complex kernels. The paper mentions using a polynomial kernel for efficiency.
- Number of Features: The paper notes that CKFD can yield more than $c-1$ discriminant features (up to $2(c-1)$), where $c$ is the number of classes. While this contributes to better performance, managing a larger number of features might require further optimization in some applications.
- Future Work: The paper implicitly suggests that the demonstrated effectiveness of CKFD on face recognition could be extended to other pattern recognition tasks where non-linear feature extraction is beneficial. The authors also mention that their approach is more transparent and simpler than existing KFD algorithms, implying potential for broader adoption and further refinement.
Summary updated