EarSet: A Multi-Modal Dataset for Studying the Impact of Head and Facial Movements on In-Ear PPG Signals

Alessandro Montanari, Andrea Ferlini, Ananta Narayanan Balaji, Cecilia Mascolo, Fahim KawsarScientific 2023 · DOI

Abstract

Photoplethysmography (PPG) is a simple, yet powerful technique to study blood volume changes by measuring light intensity variations. However, PPG is severely affected by motion artifacts, which hinder its trustworthiness. This problem is pressing in earables since head movements and facial expressions cause skin and tissue displacements around and inside the ear. Understanding such artifacts is fundamental to the success of earables for accurate cardiovascular health monitoring. However, the lack of in-ear PPG datasets prevents the research community from tackling this challenge. In this work, we report on the design of an ear tip featuring a 3-channels PPG and a co-located 6-axis motion sensor. This, enables sensing PPG data at multiple wavelengths and the corresponding motion signature from both ears. Leveraging our device, we collected a multi-modal dataset from 30 participants while performing 16 natural motions, including both head/face and full body movements. This unique dataset will greatly support research towards making in-ear vital signs sensing more accurate and robust, thus unlocking the full potential of the next-generation PPG-equipped earables.

Introduction

Photoplethysmography (PPG) is a non-invasive optical technique used to measure blood volume changes, forming the basis for heart rate and oxygen saturation monitoring in wearable devices. However, PPG signals are highly susceptible to motion artifacts (MA), which can severely degrade data quality and clinical utility. This challenge is particularly acute for "earables"—smart earbuds—where the ear canal is subject to dynamic deformations caused by head movements and facial expressions. To address the lack of specialized data for this domain, we present EarSet, a comprehensive multi-modal dataset designed to characterize and mitigate in-ear PPG motion artifacts.

System Design and Methodology

To capture high-fidelity data, the researchers developed a custom sensing platform integrated into a standard earbud form factor.

Hardware Architecture

The core of the device is a flexible PCB housing a MAXM86161 3-channel PPG sensor (supporting Green, Red, and Infrared wavelengths) and a co-located ST-LSM6DSRX 6-axis Inertial Measurement Unit (IMU). This configuration allows for the simultaneous capture of optical blood flow data and the precise motion signatures causing signal interference.

Schematic of the in-ear sensing system showing the silicon mold, flexible PCB, and PPG sensor placement within the ear canal.

Sensor Integration

The sensors are embedded within soft silicone ear tips to ensure a consistent and comfortable fit against the ear canal wall. This design mimics commercial earbuds while providing the necessary stability for research-grade data collection.

Close-up of the flexible PCB featuring the ST LSM6DSRX IMU and the MAX86161 PPG sensor.

The EarSet Dataset

The EarSet dataset consists of approximately 17 hours of synchronized multi-modal data collected from 30 participants (18 male, 12 female) with diverse skin tones (Fitzpatrick scales 1–5).

Participants performed a battery of 16 natural motions, categorized into:

  • One-shot movements: Nodding, shaking the head, winking, and blinking.
  • Continuous movements: Speaking, chewing, and full-body activities like walking and running.

Ground truth heart rate and respiration data were recorded using a medical-grade Zephyr Bioharness 3.0 ECG chest strap, providing a reliable baseline for evaluating PPG accuracy.

Key Insights and Discussion

Analysis of the EarSet data revealed several critical findings for the development of robust earables:

  • Localized Artifacts: While most facial movements significantly impact PPG signals, eye movements (horizontal and vertical) were found to cause negligible interference, likely due to the minimal involvement of periauricular muscles.
  • Morphological Distortion: Motion artifacts do not merely add noise; they alter the fundamental morphology of the PPG pulse. This distortion complicates the extraction of advanced health metrics, such as arterial stiffness, which rely on precise pulse shape analysis.
  • Stereo Sensing Advantages: Data from both ears often showed asymmetrical artifact patterns. This suggests that "stereo" earables could leverage cross-channel information to filter noise from one ear using the cleaner signal from the other.
  • Signal Quality Metrics: The study confirmed that Spectral Kurtosis is a highly effective indicator of signal quality, showing significantly higher values for clean, stationary signals compared to those corrupted by motion.

Conclusion

EarSet provides the research community with a unique resource to study the complex interactions between motion and in-ear PPG signals. By offering high-dimensional, synchronized data across a wide range of natural activities, this work supports the development of next-generation algorithms that will make ear-based cardiovascular monitoring more accurate and reliable in real-world scenarios.

Summary updated

Abstract

Photoplethysmography (PPG) is a simple, yet powerful technique to study blood volume changes by measuring light intensity variations. However, PPG is severely affected by motion artifacts, which hinder its trustworthiness. This problem is pressing in earables since head movements and facial expressions cause skin and tissue displacements around and inside the ear. Understanding such artifacts is fundamental to the success of earables for accurate cardiovascular health monitoring. However, the lack of in-ear PPG datasets prevents the research community from tackling this challenge. In this work, we report on the design of an ear tip featuring a 3-channels PPG and a co-located 6-axis motion sensor. This, enables sensing PPG data at multiple wavelengths and the corresponding motion signature from both ears. Leveraging our device, we collected a multi-modal dataset from 30 participants while performing 16 natural motions, including both head/face and full body movements. This unique dataset will greatly support research towards making in-ear vital signs sensing more accurate and robust, thus unlocking the full potential of the next-generation PPG-equipped earables.
BibTeX
@inproceedings{montanari2023earset,
  author = {Alessandro Montanari and Andrea Ferlini and Ananta Narayanan Balaji and Cecilia Mascolo and Fahim Kawsar},
  title = {{EarSet: A Multi-Modal Dataset for Studying the Impact of Head and Facial Movements on In-Ear PPG Signals}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41597-023-02762-3},
}
Figure