TL;DR
This paper introduces a novel "fixed-weight ArcFace loss" that improves facial expression recognition by explicitly constraining the angular distribution of class centers. Combined with a Vision Transformer, this approach achieves higher accuracy and computational efficiency, particularly for challenging datasets with imbalanced classes and subtle expression variations.
What Problem Does This Paper Address?
Facial expression recognition (FER) is a crucial task with applications ranging from affective computing to human-computer interaction. However, it's plagued by significant challenges: expressions within the same category can vary greatly (intra-class variance), while different expressions can look very similar (inter-class similarity). Traditional methods like Softmax loss struggle with this. While advanced loss functions like ArcFace improve inter-class separability by adding angular margins, they don't explicitly enforce a uniform distribution of class centers. This is problematic for FER datasets, which are often small, imbalanced, and contain highly correlated expression categories, leading to unstable training and biased predictions.
Methods
The core innovation is a fixed-weight ArcFace loss. This method builds upon the standard ArcFace loss by adding two key components:
- Fixed Classifier Weights: The weights of the final classification layer are initialized to be uniformly distributed on a hypersphere and are frozen during training. This prevents gradient imbalances from distorting the classifier's geometry and significantly reduces computational overhead by eliminating backpropagation through this layer.
- Angular Diversity Regularization (L4): A new regularization term is introduced to explicitly minimize the pairwise cosine similarity between class centers (weights). This encourages a more uniform angular distribution of these centers, enhancing global angular separation.
The proposed loss function, $L = L_3 + qL_4$, combines the original ArcFace loss ($L_3$) with this new angular diversity term ($L_4$), controlled by a hyperparameter $q$. This approach aims to achieve both intra-class compactness and inter-class separability, especially for minority classes.
The authors integrate this loss function with a Vision Transformer (ViT). They opt for a lightweight, custom-designed ViT architecture to effectively capture long-range dependencies and structural details crucial for FER, while also being suitable for smaller datasets and reducing computational load.
Datasets
The paper evaluates its proposed method on two widely used facial expression recognition datasets:
- FER2013: A large dataset with 35,886 grayscale images across seven basic expression categories.
- RAF-DB: A dataset with 15,339 facial expression images, also featuring seven basic emotion categories, collected under diverse real-world conditions.
Key Insights and Ideas
- The importance of class center distribution: The paper strongly argues that simply enforcing margins between individual features and their class centers (like ArcFace) isn't enough. Explicitly controlling the global angular relationships between all class centers is crucial for robust FER, especially with imbalanced data.
- Fixed weights for efficiency and stability: Freezing the classifier weights not only reduces computation but also stabilizes training. This is a clever way to leverage the benefits of margin-based losses without the full backpropagation cost.
- ViT's suitability for FER: The paper highlights how ViT's self-attention mechanism is well-suited for capturing global facial context, which is vital for understanding nuanced expressions, outperforming traditional CNNs in this regard.
- Architecture-agnostic loss: The proposed loss function demonstrates strong performance across different architectures (ViT, ResNet, VGG), suggesting its general applicability.
Key Results
- Accuracy Improvements: The proposed fixed-weight ArcFace loss consistently outperforms Softmax, Island loss, and standard ArcFace across all tested models and datasets. For instance, on RAF-DB, it achieves accuracies of 72.197% (ViT), 84.485% (ResNet18), and 82.562% (VGG19), outperforming standard ArcFace by up to 0.522%. On FER2013, it reaches 70.828% (Private) and 71.942% (Public), exceeding ArcFace by over 1.1%.
- Minority Class Performance: The method shows significant gains for minority and highly confusable classes, such as "disgust" and "fear," which are often misclassified.
- Computational Efficiency: The fixed-weight strategy reduces training time per epoch. On Transformer models, it achieves an average training time of 36.50s per epoch, which is faster than standard ArcFace (36.84s) and Island loss (39.62s).
- Robustness: The approach demonstrates consistent performance across different datasets and architectures, highlighting its generalization capability.
Limitations and Future Work
- Weight Space Overload: The authors acknowledge that as the number of expression categories increases, the available weight space might become overwhelmed.
- Future Directions: They suggest exploring dynamic weight allocation or alternative optimization strategies to address this challenge in future research.
Summary updated