Abstract
Accurate localization of facial features such as eyes and mouths is a critical prerequisite for advanced computer vision tasks, including facial recognition and expression analysis. This paper proposes a hierarchical two-stage approach for feature extraction in grayscale images. The method leverages Shannon entropy to efficiently identify eye candidates based on local texture complexity, followed by a Support Vector Machine (SVM) for precise verification. Once eyes are localized, a geometric model constrains the search space for the mouth, which is then extracted using morphological operations. Experimental results on the JAFFE and ORL databases demonstrate high detection rates and robustness against variations in expression and lighting.
Introduction
Locating facial features—specifically the corners of the eyes and the center of the mouth—serves as the foundation for higher-level tasks like gaze tracking and face recognition. However, achieving high precision on grayscale ("gray intensity") images remains challenging due to "noise" from features like eyebrows or nostrils, lighting variations, and occlusions such as glasses or facial hair. Existing methods often struggle to balance computational efficiency with the ability to distinguish between actual features and surrounding high-texture areas. This research aims to narrow the search space intelligently to achieve both speed and accuracy.
Methodology
Entropy-Based Candidate Extraction
The core innovation of this approach lies in using image entropy as a primary filter. Skin regions typically exhibit relatively uniform intensity (low entropy), whereas eye regions are characterized by high complexity and unpredictable local intensity (high entropy). By sliding a square window across the upper face and calculating Shannon entropy, the algorithm flags areas with above-average entropy as potential eye candidates. This step effectively discards approximately 80% of the face pixels, focusing subsequent processing on high-interest regions.
SVM Verification and Geometric Localization
To filter out false positives such as eyebrows, candidates are processed by a Support Vector Machine (SVM) classifier utilizing a Gaussian radial basis function (RBF) kernel. The SVM was trained on 1,200 samples, including "bootstrapped" non-eye images (e.g., nostrils) to enhance robustness.
Once the eyes are identified, they serve as stable anchors. The search for the mouth is then reduced to a localized geometric problem. A linear formula based on eye coordinates defines a specific "mouth search region." Within this region, a differential filter (the difference between Maximum and Minimum filters) isolates dark pixels. Morphological "close" operations and center-of-gravity calculations are then used to pinpoint the exact mouth center.
Experimental Results
The proposed method was evaluated using two benchmark datasets:
- JAFFE Database: 213 images of 10 female models. This dataset is relatively clean, lacking occlusions like glasses.
- ORL Database: 400 images of 40 individuals, featuring significant variations in lighting, facial hair, and expressions.
Performance was measured using a relative error metric ($err$), normalized by the inter-ocular distance. On the JAFFE dataset, the system achieved a 99.13% detection rate for eyes and 99.32% for the mouth. On the more challenging ORL dataset, the eye detection rate was 90.67%, while the mouth detection rate remained high at 97.76% (provided the eyes were correctly identified).
Discussion and Conclusion
The use of entropy as an initial filter provides a highly efficient mechanism for region-of-interest detection. By establishing the eyes as primary anchors, the system significantly simplifies the subsequent search for the mouth. While the system demonstrates high accuracy on frontal faces, performance degrades in the presence of heavy occlusions (e.g., glisten from glasses or thick mustaches) and non-frontal poses. Future work will focus on improving robustness against these occlusions and adapting the geometric models for varied head rotations.
Summary updated