PulseCam: a camera-based, motion-robust and highly sensitive blood perfusion imaging modality

Mayank Kumar, James Suliburk, Ashok Veeraraghavan, Ashutosh SabharwalScientific 2020 · DOI

TL;DR

This paper introduces PulseCam, a novel camera-based system that combines video imaging with a reference pulse oximeter signal to create high-resolution, motion-robust blood perfusion maps. PulseCam overcomes limitations of existing camera-only methods by using a brightness-invariant optical flow algorithm and a robust M-estimator, achieving significantly improved sensitivity and accuracy in detecting blood flow changes, even in challenging clinical settings.

What Problem Does This Paper Address?

Accurate measurement of blood perfusion – the delivery of oxygen and nutrients to tissues – is crucial for diagnosing and monitoring a wide range of medical conditions, from peripheral artery disease to shock. Traditional methods like pulse oximeters provide localized measurements but lack spatial information and can be insensitive to subtle changes. Emerging camera-based methods for blood perfusion imaging (photoplethysmography imaging or PPG imaging) offer spatial maps but struggle with two major challenges: motion artifacts that corrupt the data and the difficulty of recovering the weak pulsatile blood signal from strong surface reflections and measurement noise. This limits their clinical utility, especially in dynamic environments like surgery rooms.

Methods

PulseCam tackles these challenges through a multi-pronged approach:

  • Multi-Sensor Fusion: It combines video data from a camera with a reliable reference blood volume waveform from a contact-based pulse oximeter. This reference signal is crucial for robustly estimating perfusion, unlike camera-only methods that derive it from the video itself, making them susceptible to correlated noise.
  • Brightness-Invariant Optical Flow: For video stabilization, PulseCam employs a novel optical flow algorithm that is invariant to linear brightness changes. This is critical because skin appearance changes due to blood volume fluctuations, making standard brightness-constant optical flow algorithms unsuitable and leading to significant errors (20-30% in previous methods). PulseCam's approach reduces this error to below 10%.
  • Robust M-Estimator: To handle motion artifacts and outliers in the video data, PulseCam uses a robust M-estimator. This statistical approach automatically assigns low or zero weight to corrupted data points, ensuring that the perfusion estimate remains reliable even with significant noise or sudden changes in surface reflection.
  • Reference Pulse Validity Test: To further guard against corrupted reference signals, a validity test is implemented to detect sensor saturation or motion artifacts in the pulse oximeter data.

Datasets

The paper describes experiments using:

  • Lab-based studies: Controlled blood flow occlusion experiments on healthy participants with varying skin tones.
  • Clinical pilot study: Data collected from patients undergoing surgery under general anesthesia.

No new benchmark datasets were introduced; rather, existing modalities (commercial pulse oximeters and infrared thermal cameras) were used for comparison.

Key Insights and Ideas

  • Reference Signal is Key: The paper strongly emphasizes that a reliable, external reference pulse signal (from a pulse oximeter) is essential for robust blood perfusion imaging, especially in the presence of motion. Relying solely on video-derived signals makes the system vulnerable to correlated noise.
  • Brightness Invariance is Crucial for Video Stabilization: Standard optical flow algorithms fail in blood perfusion imaging because they assume constant brightness, which is violated by the very signal being measured (blood volume changes). A brightness-invariant approach is a significant improvement.
  • Microvascular Sensitivity: PulseCam's sensitivity to microvascular blood flow, due to its surface-reflection-based approach, allows it to detect subtle changes like venous or partial occlusions that are missed by deeper-penetrating contact-based methods like pulse oximeters.
  • Cost-Effectiveness: PulseCam offers a significantly cheaper alternative to expensive infrared thermography systems while providing superior performance in detecting blood flow changes.

Key Results

  • Motion Robustness: PulseCam reduces blood perfusion estimation error to below 10% in various motion scenarios, compared to 20-30% error with existing approaches.
  • Improved Sensitivity and Response Time: PulseCam demonstrates at least two times better sensitivity and three times better response time in detecting blood flow occlusions compared to infrared thermography.
  • Detection of Subtle Occlusions: PulseCam can reliably detect partial and venous blood flow occlusions, which are difficult or impossible to identify with standard pulse oximeters.
  • Clinical Validation: In a pilot study during surgery, PulseCam detected statistically significant increases in peripheral perfusion associated with anesthesia-induced vasodilation, a change not registered by a commercial pulse oximeter.
  • Cost: The system is significantly cheaper than infrared thermography.

Limitations and Future Work

  • Depth Limitation: PulseCam primarily images microvascular blood perfusion near the skin surface. It cannot image blood flow in deeper tissue layers. The authors suggest using near-infrared (NIR) cameras to potentially extend the depth of measurement.
  • Reference Signal Dependency: The system's reliability depends on the quality of the reference pulse oximeter signal. While a validity test is included, a corrupted reference signal can lead to downtime.
  • Experimental Setup: The clinical study was conducted with a prototype system in a challenging surgical environment, facing issues like unpolarized ambient light and limited control over lighting.
  • Future Directions:
    • Extending PulseCam to image other body parts (face, foot, abdomen) and internal tissues via endoscopes/laparoscopes.
  • Combining perfusion signals from multiple color channels (e.g., red, green, NIR) to create tomographic views of perfusion at different depths.
  • Developing methods to disambiguate various physiological and pathophysiological factors affecting perfusion using the high-dimensional spatial and temporal data provided by PulseCam.
  • Exploring ratios of perfusion between anatomically symmetric locations to cancel confounding factors and magnify differences, potentially for applications like assessing microcirculation in extremities or detecting critical limb ischemia.

Summary updated

Abstract

Blood carries oxygen and nutrients to the trillions of cells in our body to sustain vital life processes. Lack of blood perfusion can cause irreversible cell damage. Therefore, blood perfusion measurement has widespread clinical applications. In this paper, we develop PulseCam - a new camera-based, motion-robust, and highly sensitive blood perfusion imaging modality with 1 mm spatial resolution and 1 frame-per-second temporal resolution. Existing camera-only blood perfusion imaging modality suffers from two core challenges: (i) motion artifact, and (ii) small signal recovery in the presence of large surface reflection and measurement noise. PulseCam addresses these challenges by robustly combining the video recording from the camera with a pulse waveform measured using a conventional pulse oximeter to obtain reliable blood perfusion maps in the presence of motion artifacts and outliers in the video recordings. For video stabilization, we adopt a novel brightness-invariant optical flow algorithm that helps us reduce error in blood perfusion estimate below 10% in different motion scenarios compared to 20-30% error when using current approaches. PulseCam can detect subtle changes in blood perfusion below the skin with at least two times better sensitivity, three times better response time, and is significantly cheaper compared to infrared thermography. PulseCam can also detect venous or partial blood flow occlusion that is difficult to identify using existing modalities such as the perfusion index measured using a pulse oximeter. With the help of a pilot clinical study, we also demonstrate that PulseCam is robust and reliable in an operationally challenging surgery room setting. We anticipate that PulseCam will be used both at the bedside as well as a point-of-care blood perfusion imaging device to visualize and analyze blood perfusion in an easy-to-use and cost-effective manner.
BibTeX
@inproceedings{kumar2020pulsecam,
  author = {Mayank Kumar and James Suliburk and Ashok Veeraraghavan and Ashutosh Sabharwal},
  title = {{PulseCam: a camera-based, motion-robust and highly sensitive blood perfusion imaging modality}},
  booktitle = {Scientific},
  year = {2020},
  doi = {10.1038/s41598-020-61576-0},
}
Figure