Abstract
Human facial expressions often share overlapping physical traits, making discrete categorization challenging for traditional neural networks. This paper introduces Inter-class Relational Learning (IcRL), a novel training procedure that recognizes expressions by extracting independent expressive representations. By training on hybrid features blended from pairs of images with random ratios, the model learns the subtle relationships between different emotional classes. This approach enhances the discriminative power of learned features, significantly improving classification performance across five major benchmarks, including CK+, JAFFE, and FER2013.
The Challenge of Overlapping Emotions
Traditional Facial Expression Recognition (FER) systems typically treat emotions as isolated, discrete categories using "one-hot" labels. However, real-world expressions are rarely so distinct. For instance, "Fear" and "Surprise" often share physical characteristics such as widened eyes and an open mouth.
Because standard models ignore these intrinsic connections, the mathematical representations (feature clusters) they create often overlap, leading to misclassification of similar-looking expressions. The authors aim to solve this by forcing the network to learn the relationships between classes rather than just looking at them in isolation.
Inter-class Relational Learning (IcRL)
To address this, the proposed IcRL method functions like a "digital mixology" for features. The architecture utilizes a CNN (Inception-ResNet-v1) to extract features from pairs of different expression images. These features are then integrated using a random ratio ($\lambda$) to create a hybrid feature. An attention module assigns weights to pixels most relevant to the expressions, typically focusing on the eyes and mouth.
The network is trained to predict the exact mixing ratio of the two expressions rather than a single category, using Kullback-Leibler (KL) divergence to calculate loss. By accurately discriminating these hybrid features, the model learns to define the boundaries between classes more precisely. During inference, a single image is processed as a special case (1:0 ratio), leveraging the model's deep understanding of inter-class boundaries to make a more accurate single-class prediction.
Key Insights and Feature Separation
The primary advantage of IcRL is its ability to enlarge Fisher's criterion—the ratio of inter-class distance to intra-class distance. By forcing the model to distinguish precise ratios in hybrid features, the training procedure naturally "pushes" the feature clusters of different emotions further apart while making individual clusters more compact.
Experimental Results
The IcRL method was evaluated on five public databases: CK+, JAFFE, TFEID, BAUM-2i, and FER2013. It consistently outperformed baseline models and state-of-the-art approaches:
- CK+: Achieved 97.04% accuracy, outperforming the baseline by over 3%.
- JAFFE: Reached an impressive 99.09%, a significant jump from the 94.37% baseline.
- FER2013: Achieved 72.36% accuracy, which is highly competitive for this noisy, unconstrained dataset.
Furthermore, the model demonstrated superior cross-dataset generalization, proving that the learned features are more "universal" and less overfitted to specific dataset quirks.
Conclusion
By shifting the focus from isolated category recognition to inter-class relational learning, IcRL provides a more robust framework for facial expression recognition. It effectively handles the inherent ambiguity in human emotions, leading to more discriminative features and higher accuracy across diverse real-world scenarios.
Summary updated