Regularizing face verification nets for pain intensity regression

Feng Wang, Xiang Xiang, Chang Liu, Trac D. Tran, Austin Reiter, Gregory D. Hager, Harry Quon, Jian Cheng, Alan L. YuilleICIP 2017 · OpenReview

Abstract

Estimating facial expression intensities, particularly for automated pain assessment, is hindered by the scarcity of labeled data and significant class imbalance. This paper proposes a regularized deep regressor that leverages transfer learning from a data-rich face verification domain. By fine-tuning a state-of-the-art face verification network with a specialized regression loss and a novel application of "Center Loss" as a regularizer, the model achieves state-of-the-art performance on the UNBC-McMaster Shoulder-Pain dataset. We also introduce weighted evaluation metrics to more accurately assess performance on imbalanced clinical data.

Introduction

Automated pain assessment is a critical tool for clinical settings, offering objective monitoring that can improve anesthesia management and patient care. However, deep learning models for this task face two primary hurdles: a "Data Drought" and "Zero-Pain Bias." Gold-standard datasets like the UNBC-McMaster Shoulder-Pain dataset are small (only 200 videos from 25 patients), and over 91% of frames typically show no pain. This imbalance often leads models to converge on a trivial solution—predicting zero pain for every frame—which yields high standard accuracy but zero clinical utility.

Methodology

To overcome these challenges, we propose a framework that repurposes the rich feature representations learned by face verification networks for the task of expression intensity regression.

Transfer Learning from Face Verification

Instead of training from scratch, we initialize our model using a network pre-trained on the CASIA-WebFace dataset (0.5 million images). The hypothesis is that features optimized to distinguish between thousands of individual identities are inherently sensitive to the subtle facial muscle movements required to estimate pain intensity.

Regularized Regression Loss

We employ a Smooth L1 loss (Huber loss) for the primary regression task to ensure mathematical stability during the early stages of training. The core innovation, however, is the adaptation of Center Loss as a regularizer. While typically used in classification to pull features toward a class center, we apply it here to discrete pain levels (0–5). This forces the network to cluster its internal representations around these specific intensity levels, effectively regularizing the model against noise and preventing the regressor from producing unrealistic values.

Figure 3: Visualization of Center Loss in a 1D projection, showing how feature points are pulled toward discrete pain intensity centers (0-5).

Uniform Sampling

To address the extreme data imbalance, we implement uniform sampling during training. By ensuring the network sees high-intensity pain frames as frequently as rest frames, we prevent the model from falling into the "all-zeros" trap.

Results and Discussion

The proposed regularized deep regressor was evaluated on the UNBC-McMaster Shoulder-Pain dataset. Our method achieved a Mean Absolute Error (MAE) of 0.389 and a Pearson Correlation Coefficient (PCC) of 0.651, outperforming existing benchmarks such as OSVR and RCR.

Figure 1: Comparison of ground truth pain intensity versus model estimation across a video sequence. The model successfully tracks the onset and peak of the pain expression.

To better reflect clinical utility, we utilized weighted metrics (wMAE and wMSE). Under these metrics, our model achieved a wMAE of 0.991, a significant improvement over the baseline "all-zeros" trivial solution (wMAE of 2.143). This demonstrates that the combination of transfer learning and Center Loss regularization allows the model to truly learn the visual signatures of high-intensity pain.

Conclusion

By regularizing a face verification network with Center Loss and employing uniform sampling, we have developed a robust regressor for pain intensity. This approach effectively mitigates the issues of small sample sizes and class imbalance inherent in medical facial analysis. Future work will explore temporal modeling (e.g., LSTMs) to capture the dynamic flow of pain expressions and incorporate patient-specific baselines to account for individual subjectivity in facial expressions.

Summary updated

Abstract

Limited labeled data are available for the research of estimating facial expression intensities. For instance, the ability to train deep networks for automated pain assessment is limited by small datasets with labels of patient-reported pain intensities. Fortunately, fine-tuning from a data-extensive pre-trained domain, such as face verification, can alleviate this problem. In this paper, we propose a network that fine-tunes a state-of-the-art face verification network using a regularized regression loss and additional data with expression labels. In this way, the expression intensity regression task can benefit from the rich feature representations trained on a huge amount of data for face verification. The proposed regularized deep regressor is applied to estimate the pain expression intensity and verified on the widely-used UNBC-McMaster Shoulder-Pain dataset, achieving the state-of-the-art performance. A weighted evaluation metric is also proposed to address the imbalance issue of different pain intensities.
BibTeX
@inproceedings{wang2017regularizing,
  author = {Feng Wang and Xiang Xiang and Chang Liu and Trac D. Tran and Austin Reiter and Gregory D. Hager and Harry Quon and Jian Cheng and Alan L. Yuille},
  title = {{Regularizing face verification nets for pain intensity regression}},
  booktitle = {ICIP},
  year = {2017},
  url = {https://openreview.net/forum?id=oBJ2vkTH-7},
}
Figure