Video Compression Artifacts in Remote Photoplethysmography

18 concepts12 papers1 proposal

When your heart beats, it pumps blood throughout your body, causing rhythmic volume changes in the blood vessels of your face. Although invisible to the naked eye, these changes alter the amount of light reflected by your skin. Remote photoplethysmography (rPPG) is the science of using standard webcams or smartphone cameras to detect these minute color fluctuations, allowing for non-contact monitoring of heart rate and heart rate variability (HRV).

However, deploying rPPG in the real world reveals a fundamental engineering conflict: the very algorithms we use to transmit and store video across the internet are mathematically designed to destroy the exact signals rPPG relies on.

The Perceptual Coding Conflict

To understand why video compression breaks rPPG, we have to look at how modern video codecs work. Codecs like H.264 (x.264) and H.265 (x.265) are built around perceptual coding. Their primary goal is to minimize file size while keeping the video visually indistinguishable to a human observer.

As Christian S. Pilz et al. point out in Local Group Invariance for Heart Rate Estimation From Face Videos in the Wild, human vision is highly insensitive to high-frequency, low-amplitude spatial and temporal color variations. Consequently, compression algorithms intentionally discard this "redundant" data.

Unfortunately, as Jeremy Speth et al. note in Non-Contrastive Unsupervised Learning of Physiological Signals from Video, the blood volume pulse manifests as a tiny 1–2% variation in pixel intensity. Because this physiological signal lives entirely within the low-amplitude, high-frequency spectrum that codecs discard, standard compression treats the human heartbeat as visual noise and systematically erases it.

[Raw Video] ---> [Lossy Codec (Quantization & Smoothing)] ---> [Compressed Video]
  Contains          Discards low-amplitude, high-frequency          Heartbeat signal
  1-2% pulse        color variations to save bandwidth              is flattened or
  fluctuations                                                      heavily distorted

How Compression Destroys the Pulse

Compression degrades the physiological signal through two primary mechanisms: spatial-temporal smoothing and quantization.

Spatial-Temporal Smoothing

Codecs group similar pixels into blocks and average them over time to save bandwidth. Radim Spetlík et al. demonstrate in Visual Heart Rate Estimation with Convolutional Neural Network that this high-level smoothing acts as a severe bottleneck for rPPG. By blending minute pixel-level variations across space and time, the codec flattens the subtle temporal waveforms required for pulse detection. Amogh Gudi et al. echo this in Efficient Real-Time Camera Based Estimation of Heart Rate and Its Variability, showing that this smoothing makes it incredibly difficult for traditional signal processing methods to extract accurate HRV metrics, which require precise timing between beats.

Quantization Artifacts

Quantization maps a continuous range of color values into a smaller set of discrete steps. When a video is heavily compressed, the step size becomes larger than the 1–2% pulse signal itself. As a result, the subtle gradient of a heartbeat is rounded off into static blocks. John Gideon et al. explain in The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video that these quantization artifacts introduce artificial edges and distortions that completely obscure the underlying blood volume pulse.

The mathematical reality of this degradation is stark. In iPhys: An Open Non-Contact Imaging-Based Physiological Measurement Toolbox, Ethan B. Blackford demonstrates that standard video codecs cause a linear degradation in the signal-to-noise ratio (SNR) of recovered blood volume pulse signals. As bitrate drops, the physiological signal vanishes in a predictable, destructive trajectory.

The Hidden Culprit: Spatial Interpolation

The loss of signal is not exclusively driven by video codecs; standard preprocessing pipelines also introduce corruption. To feed facial videos into deep neural networks, researchers typically standardize input dimensions by resizing frames.

However, in Learning Motion-Robust Remote Photoplethysmography through Arbitrary Resolution Videos, Jianwei Li et al. reveal that simple spatial interpolation algorithms (like bilinear or bicubic resizing) inadvertently corrupt high-frequency physiological cues embedded in pixel data. When pixels are mathematically blended to fit a new resolution, the delicate, localized phase information of the blood flow is blurred, compounding the damage already inflicted by lossy compression.

Downstream Bottlenecks for Machine Learning

This signal degradation creates severe roadblocks for modern deep learning approaches to rPPG, particularly in three areas:

  1. Unsupervised Learning: Large-scale, self-supervised models rely on massive, uncurated datasets (like web videos) to learn representations without medical ground truth. However, Jeremy Speth et al. note that because web content is heavily compressed, the physiological signal is often entirely missing from the source data, preventing unsupervised models from learning meaningful features. Similarly, H. Wang et al. show in Self-Supervised Representation Learning Framework for Remote Physiological Measurement Using Spatiotemporal Augmentation Loss that high compression combined with small dataset sizes masks the high-fidelity temporal information needed for robust signal reconstruction.
  2. Deepfake Detection: Security frameworks often use rPPG to detect face forgeries, operating on the premise that AI-generated faces lack a real biological pulse. However, Jiahao Liang et al. demonstrate in Identifying Rhythmic Patterns for Face Forgery Detection and Categorization that the heavy compression applied by social media platforms destroys these rhythmic patterns. This makes it incredibly difficult to distinguish a compressed real video from a compressed fake video based on physiology alone.
  3. The Real-World Performance Gap: Models trained on pristine, uncompressed laboratory data experience a sharp decline in accuracy when deployed in real-world settings. As Puneet Gupta et al. observe in MOMBAT: Heart Rate Monitoring from Face Video using Pulse Modeling and Bayesian Tracking, this creates a massive performance gap between high-quality reference streams and the lower-quality, compressed video streams typical of telehealth or mobile applications.

Engineering Around the Compression Barrier

To bridge this gap, recent research focuses on building architectures that are explicitly aware of compression artifacts.

One prominent strategy is neural video enhancement. In Remote Heart Rate Measurement From Highly Compressed Facial Videos: An End-to-End Deep Learning Solution With Video Enhancement, Wei Peng et al. introduce a Spatio-Temporal Video Enhancement Network (STVEN) paired with a specialized recovery network (rPPGNet). Instead of trying to extract a pulse from a degraded video directly, the framework uses a learned upsampling and de-blocking step to mathematically "reconstruct" the discarded low-amplitude color variations before performing rPPG extraction.

[Compressed Video] ---> [STVEN (Enhancement)] ---> [Restored Cues] ---> [rPPGNet] ---> [Accurate HR]

Alternatively, researchers are advocating for a shift in how data is captured. In introducing the MMPD: Multi-Domain Mobile Video Physiology Dataset, Jiankai Tang et al. emphasize that standard mobile video compression remains a primary barrier to non-contact health monitoring. They suggest that future breakthroughs will rely on developing specialized, physiology-preserving codecs or utilizing raw, uncompressed video formats directly at the point of capture.

Key Foundational Work

Visual Heart Rate Estimation with Convolutional Neural Network (2018) — contributed 3 concept elements to this direction.
iPhys: An Open Non-Contact Imaging-Based Physiological Measurement Toolbox (2019) — contributed 3 concept elements to this direction.
Local Group Invariance for Heart Rate Estimation From Face Videos in the Wild (2018) — contributed 2 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 11 papers
  2. Version 1 · 2026-10-04 · 22 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_compression_induced_signal_loss_66cbc46f,
  title = {compression-induced signal loss},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compression-induced-signal-loss/rdr_66cbc46fa7ce4cce83c986c41989b97c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 2, frozen text):

@misc{lacuna_compression_induced_signal_loss_66cbc46f_v2,
  title = {compression-induced signal loss},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compression-induced-signal-loss/rdr_66cbc46fa7ce4cce83c986c41989b97c/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake