Remote Photoplethysmography for Highly Compressed Video

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 222 papers36 conceptsfrozen 2026-10-04

Remote photoplethysmography (rPPG) is the science of measuring physiological signals—like heart rate and respiration—using nothing but a standard camera. It works by detecting the minute, rhythmic changes in skin color caused by the blood volume pulse. To the human eye, these changes are invisible, often representing less than a 1% variation in pixel intensity.

While deep learning has made rPPG increasingly viable, it faces a fundamental adversary: the video compression algorithms that power the modern web.

The Conflict Between Compression and Physiology

Standard video codecs like H.264 or HEVC are designed for human consumption. They operate on a simple principle: if a human can’t see it, throw it away. As Daniel McDuff explains, these algorithms are optimized to discard subtle pixel fluctuations and high-frequency temporal data to save bandwidth.

For rPPG, this is catastrophic. Radim Spetlík and Jan Čech identify video compression as the primary bottleneck for remote sensing. When a video is compressed, the subtle 1–2% intensity changes necessary for pulse detection are smoothed out or quantized into oblivion. This creates a "physiological information gap" where the visual scene looks perfect to a person, but the underlying biological data has been erased.

Why Traditional Enhancement Fails

One might assume that standard video enhancement—making a video look "sharper" or "cleaner"—would help. However, Wei Peng and Guoying Zhao argue that traditional metrics like Peak Signal-to-Noise Ratio (PSNR) are misleading in this context.

A video can have a high PSNR and look aesthetically pleasing while still having its physiological signals destroyed. In fact, many "denoising" filters actually worsen rPPG performance because they treat the tiny pulse-induced color shifts as noise, smoothing them out to create a more pleasing, stable image.

Recovering the Invisible Signal

To solve this, researchers are moving away from general-purpose video processing toward architectures that prioritize "physiological fidelity."

  1. Spatio-Temporal Enhancement: Wei Peng and Zitong Yu developed the Spatio-Temporal Video Enhancement Network (STVEN). Instead of making the video look better for humans, STVEN is trained specifically to "recover" the rhythmic color variations lost during compression, which are then fed into a recovery network called rPPGNet.
  2. Feature Extraction Resilience: Jianwei Li demonstrated that specialized feature extraction modules can extract pulse signals even from heavily artifacted video, reducing heart rate estimation error by over 75% compared to standard models. This suggests that the signal isn't always gone; it's often just buried in a way that standard CNNs cannot see.
  3. Cross-Modal Restoration: In the realm of respiratory monitoring, Alicja Kwasniewska showed that Transformers can be used to transfer high-resolution texture details from visible light images to low-resolution thermal video, restoring the fine-grained temporal dynamics needed to track breathing.

Learning Without Labels

The compression problem also hinders the scaling of these models. Most deep learning requires "ground truth" data (video synced with an ECG). While the internet is full of facial video, Jeremy Speth and Adam Czajka point out that lossy compression on web platforms prevents unsupervised models from learning from this massive data pool.

To bypass this, researchers like H. Wang are using self-supervised learning with "spatiotemporal augmentations." By teaching models to be invariant to the types of distortions caused by compression and movement, they can learn to find the pulse in "dirty" data without needing a medical-grade reference for every frame.

The Path Forward

The consensus across recent research is that rPPG cannot treat video as a static input. Whether it is Amogh Gudi focusing on real-time efficiency or Ali Etemad working on smart environments, the goal is the same: building systems that understand the physics of skin reflection well enough to see through the "fog" of digital compression. Success in this field would turn every compressed Zoom call or YouTube stream into a potential health monitoring tool.

Papers in this version

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 11 papers
  2. Version 1 · 2026-10-04 · 22 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_compression_induced_signal_loss_66cbc46f,
  title = {compression-induced signal loss},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compression-induced-signal-loss/rdr_66cbc46fa7ce4cce83c986c41989b97c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_remote_photoplethysmography_for_highly_c_66cbc46f_v1,
  title = {Remote Photoplethysmography for Highly Compressed Video},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/remote-photoplethysmography-for-highly-compressed-video/rdr_66cbc46fa7ce4cce83c986c41989b97c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}