Biometric systems, such as face and iris scanners, are increasingly prevalent in security and identification. However, their performance often degrades significantly when the target is partially obscured by masks, sunglasses, or hair. This paper, presented at AAAI-20, introduces a novel framework to address this "global" weakness of standard Convolutional Neural Networks (CNNs) by combining feature extraction with structural graph analysis.
The Challenge of Occlusion in Biometrics
Modern computer vision relies heavily on CNNs, which extract global feature vectors for identification. When an object is partially occluded—for instance, a mask covering a face or an eyelid covering an iris—the "missing" data distorts these feature vectors, leading to recognition failures.
Existing solutions typically attempt to detect occlusions first and then mask those areas. However, this approach is prone to error propagation; if the initial mask detection is inaccurate, the subsequent recognition is compromised. Furthermore, standard CNNs do not explicitly model the spatial relationships between different parts of a face or iris, which is vital information when some parts are unavailable.
Proposed Method: Dynamic Graph Representation (DGR)
To overcome these limitations, the researchers developed Dynamic Graph Representation (DGR). This framework represents biometric images as graphs of interconnected parts, allowing the system to adaptively ignore unreliable regions during the matching process.
The DGR framework consists of four primary components:
- Graph Generator: This module identifies $N$ key spatial regions (nodes) within the CNN's feature maps. It constructs a "Feature Graph" where nodes represent local parts and edges represent their spatial relationships.
- SE-GAT (Squeeze-and-Excitation Graph Attention Network): A custom GNN layer that uses "Squeeze-and-Excitation" to highlight important feature channels and "Graph Attention" to weigh the significance of connections between different biometric parts.
- Dynamic Graph Matching: During the comparison of two images, the system calculates the similarity between corresponding nodes. If a pair of nodes is highly dissimilar—indicating an occlusion—the system adaptively prunes those nodes from the graph before the final decision.
- Graph Triplet Loss: A training objective that ensures the model minimizes differences in both semantic features (appearance) and structural features (arrangement).
Key Insights and Results
- Pruning is better than Masking: DGR identifies occlusions during the matching process by spotting outliers in node similarity, rather than relying on a pre-processing mask detection step.
- Spatial Relationships Matter: By explicitly modeling edges, the system retains a sense of geometry even when nodes are removed. This makes the AI's decision-making more transparent and interpretable.
- Superior Accuracy: The DGR framework outperformed state-of-the-art baselines significantly. On the ND-LG4000 iris dataset, the Equal Error Rate (EER) dropped from 1.77% to 0.62%. In face verification with 30% occlusion, DGR achieved a True Positive Rate (TPR) of 71.8%, compared to just 22.47% for a standard Light CNN.
Conclusion
By fusing the feature-extraction power of CNNs with the structural flexibility of Graph Neural Networks, the DGR framework provides a robust solution for biometric identification under occlusion. While matching graphs is more computationally intensive than simple vector comparisons, the significant gains in accuracy and reliability make it a promising direction for future high-security applications.
Summary updated