Automatic Facial Paralysis Assessment via Computational Image Analysis

Chaoqun Jiang, Jianhuang Wu, Weizheng Zhong, Mingqiang Wei, Jing Tong, Haibo Yu, Ling WangJournal of Healthcare Engineering 2020 · DOI

TL;DR

This paper introduces an automated system for assessing facial paralysis (FP) severity using computational image analysis. By analyzing facial blood flow patterns captured with Laser Speckle Contrast Imaging (LSCI), the system aims to provide an objective and quantitative alternative to subjective clinical assessments, achieving high accuracy.

What Problem Does This Paper Address?

Diagnosing and grading facial paralysis, a condition resulting from damage to the facial nerve, has traditionally relied on subjective methods like the House-Brackmann (HB) grading system. This system depends heavily on the clinician's expertise, leading to potential inconsistencies and a lack of objective, quantitative data. While objective methods like electromyography (EMG) exist, they can be invasive or complex. The paper addresses the need for a non-invasive, objective, and quantitative assessment tool that can reliably determine the severity of facial paralysis.

Methods

The proposed approach involves several key steps:

  1. Facial Blood Flow Measurement: Laser Speckle Contrast Imaging (LSCI) is used to capture both standard RGB color images and blood flow images of the patient's face. LSCI is a non-invasive technique that measures microcirculation perfusion by analyzing the blurring of interference patterns caused by moving blood cells.
  2. 3D Face Reconstruction and Segmentation:
    • Facial Landmark Fitting: Key facial landmarks (68 points) are identified in the color image.
  3. Camera Estimation: A camera model is estimated using these landmarks and their corresponding 3D model points.
  4. 3D Model Fitting: A 3D morphable face model (Surrey face model) is fitted to the 2D landmarks using PCA coefficients to reconstruct a 3D face model.
  5. Region Segmentation: Predefined regions of interest (e.g., eyebrow, eye circumference, cheek) are mapped from the 3D model onto the 2D facial blood flow image.
  6. Facial Blood Flow Distribution Analysis: The average blood flow volume within each segmented region is calculated. The ratio of blood flow on the affected side to the healthy side is used as a key indicator, as asymmetry is expected in FP.
  7. HB Score Classification: Machine learning classifiers (K-Nearest Neighbors (K-NN), Support Vector Machine (SVM), and Neural Network (NN)) are trained using the extracted facial blood flow distribution characteristics to predict the HB score.

Datasets

The study utilized a dataset of 80 participants, including 8 healthy individuals and 72 patients with facial paralysis. The data consisted of 8,000 LSCI images (100 per participant). For evaluating the segmentation performance, 100 facial blood flow images were manually labeled by a skilled clinician to serve as ground truth. The HB scores of the patients were obtained from experienced clinicians.

Key Insights and Ideas

  • Blood Flow as a Biomarker: The core idea is that facial skin perfusion is directly affected by facial nerve damage and its severity, making it a reliable indicator for assessing FP.
  • Bridging Subjectivity and Objectivity: The system aims to transform the subjective HB grading system into an objective, automated process, enhancing diagnostic consistency.
  • Leveraging LSCI for Non-Invasive Assessment: LSCI's ability to provide real-time, full-field blood flow imaging without requiring needles or invasive procedures is crucial for patient comfort and ease of use.
  • 3D Reconstruction for Precise Segmentation: Using 3D face reconstruction allows for accurate mapping of predefined facial regions onto the 2D blood flow images, ensuring that specific areas relevant to FP are analyzed.

Key Results

  • High Segmentation Accuracy: The facial blood image segmentation method achieved an overall accuracy of 93.98% (using Dice's coefficient). Specific regions like the cheek and eye circumference showed accuracies above 95%.
  • Optimal Classifier Performance: The Neural Network (NN) classifier achieved the highest HB classification accuracy of 97.14%. SVM and K-NN performed with accuracies of 87.06% and 69.55% (mean values), respectively.
  • Clinician Agreement: Experimental evaluations demonstrated that the automated assessment results highly aligned with those obtained by experienced clinicians.
  • Correlation with Severity: The study found that the asymmetry in facial blood flow distribution, particularly in orbital regions, increased with higher HB scores, supporting the hypothesis that blood flow reflects FP severity.

Limitations and Future Work

  • Dataset Size: The authors acknowledge that the dataset of 80 volunteers, while substantial, is considered relatively small for training very large deep learning networks.
  • Future Work: The paper suggests that the proposed LSCI-based automated system is a convenient tool for both patients and clinicians and is the first of its kind to apply LSCI for automated facial paralysis assessment. Further research could explore larger datasets and more complex deep learning architectures.

Summary updated

Abstract

Facial paralysis (FP) is a loss of facial movement due to nerve damage. Most existing diagnosis systems of FP are subjective, e.g., the House-Brackmann (HB) grading system, which highly depends on the skilled clinicians and lacks an automatic quantitative assessment. In this paper, we propose an efficient yet objective facial paralysis assessment approach via automatic computational image analysis. First, the facial blood flow of FP patients is measured by the technique of laser speckle contrast imaging to generate both RGB color images and blood flow images. Second, with an improved segmentation approach, the patient's face is divided into concerned regions to extract facial blood flow distribution characteristics. Finally, three HB score classifiers are employed to quantify the severity of FP patients. The proposed method has been validated on 80 FP patients, and quantitative results demonstrate that our method, achieving an accuracy of 97.14%, outperforms the state-of-the-art systems. Experimental evaluations also show that the proposed approach could yield objective and quantitative FP diagnosis results, which agree with those obtained by an experienced clinician.
BibTeX
@article{jiang2020automatic,
  author = {Chaoqun Jiang and Jianhuang Wu and Weizheng Zhong and Mingqiang Wei and Jing Tong and Haibo Yu and Ling Wang},
  title = {{Automatic Facial Paralysis Assessment via Computational Image Analysis}},
  journal = {Journal of Healthcare Engineering},
  year = {2020},
  doi = {10.1155/2020/2398542},
}
Figure