A Perifacial EMG Acquisition System for Facial-Muscle-Movement Recognition

Jianhang Zhang, Shucheng Huang, Jingting Li, Yan Wang, Zizhao Dong, Su-Jing Wang2023 · DOI

TL;DR

This paper introduces a novel, low-cost, multi-channel wireless EMG system designed for capturing facial muscle movements. By placing electrodes around the face's periphery rather than directly on muscles, the system overcomes limitations of existing methods, enabling more natural movement and achieving high accuracy in recognizing five distinct facial actions.

What Problem Does This Paper Address?

Facial muscle movements are crucial for nonverbal communication, emotion recognition, and human-computer interaction. Traditional methods for capturing these movements, like facial image analysis, are sensitive to environmental factors such as lighting and camera angles. While electromyography (EMG) offers a more direct measure of muscle activity, existing EMG acquisition systems often suffer from high costs, limited channels, and cumbersome setups that can inhibit natural facial expressions. Specifically, placing electrodes directly on facial muscles can cause discomfort, weight the skin, and potentially lead to signal interference or inaccurate readings due to electrode displacement. This paper aims to address these limitations by developing a more practical, flexible, and accurate system for facial EMG acquisition and movement recognition.

Methods

The core of this work is a custom-designed, multi-channel wireless EMG acquisition system. Key features include:

  • Perifacial Electrode Placement: Instead of placing electrodes directly on facial muscles, this system uses a "distal arrangement" where 16 electrodes are placed around the periphery of the face. This approach aims to capture crosstalk signals propagated from muscle sources via volume conduction, minimizing interference with natural facial movements.
  • Hardware Architecture: The system comprises an acquisition node and a receiver node. The acquisition node includes:
  • Signal-Acquisition Module: Utilizes ADS1299 chips for multi-channel (up to 16 channels in this design, expandable) analog-to-digital conversion with 24-bit resolution and a 1 KHz sampling rate. It supports both unipolar and bipolar acquisition modes.
  • Microcontroller Unit (MCU): An STM32F429 controls the acquisition module and wireless transmitter.
  • Wireless Transmitter: An ESP32C3-MINI1 module for Wi-Fi transmission of data to the receiver.
  • Power-Management Module: A lithium battery powers the system for portability.
  • Software System: Includes a hardware program for data acquisition and transmission, and a graphical user interface (GUI) developed in C++ using the QT framework for real-time data reception, processing, storage, and display.
  • Data Pre-processing: Standard signal processing techniques are applied, including notch filtering (50Hz), bandpass filtering (20-350Hz), DC removal, full-wave rectification, and low-pass filtering for signal envelope smoothing.
  • Source Separation: Independent Component Analysis (ICA) is employed to disentangle mixed signals captured by the perifacial electrodes, aiming to identify contributions from individual muscle sources.
  • Feature Extraction: A combination of time-domain (IEMG, VAR, MAV, SSI, RMS) and frequency-domain (FC, MF, FRMS) features are extracted from the ICA components.
  • Classification: Three machine learning classifiers were evaluated: Support Vector Machine (SVM), Random Forest, and Backpropagation Neural Network (BPNN).

Datasets

The paper does not explicitly mention pre-existing public datasets for facial EMG recognition. Instead, the authors collected their own dataset.

  • Custom Dataset: 484 EMG signal samples were collected from ten healthy subjects performing five facial-muscle movements (raising eyebrows, frowning, closing eyes, pursing lips, lifting corners of the mouth) at two intensities (normal and forceful). Video recordings were used to aid in labeling the EMG data.

Key Insights and Ideas

  • Perifacial Placement is Viable: The most significant insight is that placing EMG electrodes around the face's periphery, rather than directly on muscles, can still yield reliable signals for facial movement recognition. This approach significantly enhances user comfort and naturalness.
  • Distal Signals Carry Useful Information: Despite being "crosstalk" signals, the electrical activity detected at the face's edge is sufficiently informative to distinguish between different facial movements.
  • Random Forest Outperforms: For facial-muscle-movement recognition, the Random Forest classifier demonstrated superior performance compared to SVM and BPNN, achieving the highest accuracy.
  • System Reliability: The proposed system shows good consistency with a commercial EMG device, with correlation coefficients above 70% across various facial movements, validating its reliability.
  • Flexibility and Scalability: The system's design allows for easy expansion of channels and offers flexibility in acquisition modes (unipolar/bipolar), making it adaptable for various research and application needs.

Key Results

  • Reliability Comparison: The proposed system achieved correlation coefficients above 70% (Spearman, Energy Ratio, LCC, CCC) when compared to a commercial BIOPAC MP160 system across four facial movements. For example, for "Frown," the LCC was 0.88 and CCC was 0.88.
  • Facial Movement Recognition (Five Classes - Forceful):
    • Random Forest: 91.79% accuracy
    • SVM: 82.14% accuracy
    • BPNN: 64.28% accuracy
  • Facial Movement Recognition (Ten Classes - Normal & Forceful):
    • Random Forest: 77.31% accuracy
    • SVM: 71.13% accuracy
    • BPNN: 52.57% accuracy
  • Best Classifier: Random Forest consistently provided the best classification results across both the five-class and ten-class recognition tasks.

Limitations and Future Work

  • Signal Strength: The perifacial electrode placement, while improving comfort, can result in weaker signals, making it challenging to capture subtle movements like normal blinking or frowning.
  • Data Volume and Transmission: While the system supports up to 64 channels, Wi-Fi transmission speed (limited to 2.9 Mbps/s) can be a bottleneck for very high channel counts or sampling rates.
  • Environmental Challenges: Real-world applications may face issues with diverse environments, electromagnetic interference, and hardware performance degradation.
  • Future Work:
    • Investigate the use of smaller, more flexible electrodes to potentially acquire finer signals.
  • Further explore the optimal distance of electrodes from facial-muscle sources.
  • Integrate the system with computer vision for multimodal analysis, enhancing facial-muscle-movement detection and recognition.
  • Address challenges related to environmental diversity and signal robustness through advanced data processing and algorithm optimization.
  • Explore better wireless transmission solutions for higher data throughput.

Summary updated

BibTeX
@misc{zhang2023perifacial,
  author = {Jianhang Zhang and Shucheng Huang and Jingting Li and Yan Wang and Zizhao Dong and Su-Jing Wang},
  title = {{A Perifacial EMG Acquisition System for Facial-Muscle-Movement Recognition}},
  year = {2023},
  doi = {10.3390/s23218758},
}
Figure