Acquisition Fingerprints for Self-Configuring Camera Pulse Estimation Across Sites

Measuring heart rate from an ordinary video of a face, called remote photoplethysmography (rPPG), works well inside the lab that collected the training data and degrades sharply when the camera, compression or recording site changes. We propose to borrow the main idea behind self-configuring medical segmentation. Before training, those systems read simple properties of a dataset, such as voxel spacing and image size, and use fixed rules to choose preprocessing, network shape and patch size. Our version reads the temporal properties of a new camera's footage: its true frame timing, compression strength, row-by-row readout delay, noise floor, and how strong the pulse signal is in skin pixels. From these it chooses the video pipeline's settings automatically. All of these properties can be estimated from unlabeled video, so a new hospital or phone can be fingerprinted on arrival with no contact sensor. The same fingerprint also yields a measure of how far the new site sits from anything seen in training.

The claim is falsifiable. Most of the gap between a default rPPG configuration and one tuned on the target site's labels can be recovered from an unlabeled acquisition fingerprint alone. The rules that map fingerprint to configuration can be derived on synthetically degraded versions of clean datasets and then hold on real, unseen devices. If the rules only work on the synthetic degradations, or if a simpler augmentation baseline closes the same gap, the idea fails, and the experiments are designed to show either outcome.

Core Claim and Where It Sits

Self-configuring systems such as nnDetection and nnLandmark extract a dataset "fingerprint" and turn it into a pipeline through deterministic rules. One example: downsampling is restricted along coarse axes, as motivated by MNet. Another: patch size is set so the receptive field covers the target's physical extent. AdwU-Net found that the best depth and width differ by task. All of these supply the method, but every fingerprint they compute is spatial and static. In rPPG the physical quantity that sets the right receptive field is time: how many cardiac cycles a window spans at the frame rate actually delivered.

The motivating evidence comes from four papers. Promoting Generalization in Cross-Dataset Remote Photoplethysmography shows that models memorize the training site's heart-rate band, and that speed and modulation augmentation cut cross-dataset error from 13.69 to 2.25 BPM. Its architecture is still fixed, and it fails on spatial and motion artifacts. Remote Photoplethysmography: Rarely Considered Factors measures three effects. Rolling shutter, where rows are read out one after another, introduces a phase delay of about 0.02 s, roughly 10° at cardiac frequencies. Frame-timing jitter is negligible. The choice of evaluation window shifts the computed heart rate by 4–10 bpm. That last finding means window length is a configuration choice with large consequences, not a detail. MMPD shows supervised models collapse on compressed phone video, darker skin and walking.

The closest adaptive work is PRISM. It re-tunes two scalars on the extracted 1D color trace every 10 seconds: a color-mixing weight and a detrending strength. We differ in two ways. Configuration happens once per site from acquisition physics rather than per window from the output signal. And it controls what PRISM cannot reach: temporal resampling, window length, the network's temporal receptive field, spatial pooling scale and augmentation ranges.

The ingredient that most separates this proposal from "nnU-Net for rPPG" is how the rules are obtained. With only four or five public rPPG datasets, rules fitted directly to datasets would overfit. Instead we build hundreds of synthetic "sites" from clean recordings and fit the rules there.

Research Program

Fingerprint. For each site we estimate the following from unlabeled clips:

  • effective frame rate and timing regularity, from container timestamps checked against inter-frame motion;
  • compression strength, from blockiness and bitrate;
  • rolling-shutter skew, from row-dependent phase of light flicker where it is present, otherwise from the device model;
  • exposure blur, from motion-streak width during head movement;
  • skin-pixel noise floor and pulsatile signal-to-noise ratio, from the spectral peak prominence of a training-free extractor such as POS from the rPPG-Toolbox;
  • face size in pixels;
  • the head-motion spectrum.

Configuration space. The fingerprint determines:

  • the temporal resampling target;
  • the analysis window length, set to a minimum number of cycles at the lowest plausible heart rate;
  • the temporal kernel and stride schedule, so the receptive field covers that window without aliasing the upper heart-rate band;
  • the band-pass limits;
  • spatial pooling resolution, with coarser pooling when per-pixel signal-to-noise is low, a likely relevant knob for darker skin;
  • whether difference frames are used;
  • the ranges for speed, compression and blur augmentation.

Backbones come from the toolbox (PhysNet, PhysFormer) so that architecture is not confounded with configuration.

Synthetic site generator and rule fitting. From PURE and UBFC-rPPG we generate degraded variants. Frames are dropped or temporally resampled to lower rates. Video is re-encoded at controlled bitrates. Rolling shutter is simulated by interpolating each row toward the next frame in proportion to its row index. Neighbouring frames are averaged to mimic longer exposure. Sensor noise is added and faces are downscaled. Averaging frames produces stepping rather than true continuous blur, as Nah et al. and Cao et al. warn, so we use it only for mild exposure increases and treat blur rules as the least trusted. For each synthetic site we grid-search the configuration space using that site's labels, giving the "oracle" setting. We then fit simple, inspectable rules from fingerprint to oracle configuration: thresholds and monotone maps of the kind nnU-Net uses, not a black-box regressor.

Out-of-distribution score. We define a distance $d$ between a new site's fingerprint and the training fingerprints, measured in standardized units with the Mahalanobis form. We test whether $d$ predicts per-site error before any labels exist. This addresses the open problem that deployments currently cannot tell at inference time how far a site's sensor deviates from the training data.

Rolling-shutter rule. Rolling shutter matters little for heart-rate frequency, since a constant phase shift does not move the spectral peak. It does break the assumption, in contrastive methods such as Contrast-Phys, that forehead and cheek pulse in sync. When estimated skew is large, the configurator either pairs regions from similar rows or phase-corrects them. We expect this to show up in waveform correlation and inter-region phase, not in heart-rate error, and we will report it that way rather than claim heart-rate gains.

Baselines

The naive reference is the rPPG-Toolbox default for each backbone: resample to a fixed rate, fixed window, fixed architecture. It is trained on the source datasets and applied unchanged to the target. As a reference point, the toolbox reports PhysNet with normalized inputs at MAE 1.29 ± 1.50 on its benchmark setting, and PhysLLM reports a PhysNet baseline of 4.80 bpm MAE on MMPD.

The strongest learned competitor is speed and modulation augmentation from Promoting Generalization. It reports cross-dataset MAE of 2.25 BPM from PURE to UBFC-rPPG. We reimplement it on the same backbones, same source data, same training epochs, and the same hyperparameter-search budget we allow our configurator's backbone. Our method counts as beating it only if the configurator is significantly lower in MAE under leave-one-dataset-out with augmentation enabled in both arms. Comparing configuration without augmentation against augmentation would be unfair.

The strongest training-free competitor is PRISM, which reports MAE 0.77 bpm on PURE and 0.66 on UBFC-rPPG. Both are training-free numbers, not cross-dataset ones, and they set a demanding floor on clean video. We run it on every target, including MMPD, where no reported PRISM result was found on Lacuna.

On MMPD the reported reference points are PhysLLM at 4.36 bpm and TS-CAN at 15.43 bpm for skin type 6. We cite PhysLLM without rerunning it, because its 1.5B-parameter language model is outside our compute budget, and we state that the protocols may differ.

The upper bound is the per-target oracle configuration, chosen with target labels. A random-configuration baseline, drawn from the same space, tests whether any reconfiguration helps.

Ablations isolate the key ingredient. First, rules fitted on real source datasets only, versus rules fitted on the synthetic site grid. Second, the configurator with one fingerprint entry removed at a time. Third, the full configurator without the rolling-shutter pairing rule.

Why not just…?

Why not just use speed augmentation and a fixed pipeline? This is the real threat. Most cross-dataset error in Promoting Generalization came from heart-rate memorization, which augmentation fixes. Our bet is that the remaining error, on compressed, low-SNR phone video such as MMPD, comes from window length, pooling scale and frequency bands that augmentation does not touch. The augmentation-plus-configurator versus augmentation-only comparison on MMPD settles this. If the difference is within noise, the configurator idea is not worth its complexity, and we would report that.

Why not just run PRISM? PRISM tunes two scalars after the signal has been extracted. It cannot change how pixels are pooled or how long a window is, and the window-size effect alone moves heart rate by 4–10 bpm according to Rarely Considered Factors. It was also evaluated only on lighter skin under mild motion. Running it on MMPD's skin-type and walking subsets shows directly whether per-window scalar tuning already captures what site-level configuration offers.

Why not just fine-tune on a few labeled clips from the new site? This is a fair alternative whenever a contact oximeter is available for a short calibration. We include it as a second upper-bound arm with 1, 3 and 10 labeled minutes. Our argument is about the regime where no contact sensor is available at deployment. Fine-tuning also produces no warning when a site is far from training, which the fingerprint distance does.

Why not just route features into a large pretrained model? PhysLLM reaches 4.36 bpm on MMPD but needs a 1.5B-parameter language model plus LLaVA prompts, which rules out phone or bedside use. We position our method as a low-compute alternative and do not claim to beat that number unless we actually do.

Proposed Experiments

Sites and protocol. Real sites are PURE, UBFC-rPPG, UBFC-Phys and MMPD, with SUMS (thuhci/SUMS) as an additional unseen camera setup. We use leave-one-dataset-out, with rules fitted only on synthetic degradations of the non-held-out clean sets. Metrics are MAE, RMSE and Pearson correlation of heart rate from the toolbox. MMPD results are stratified by skin type, lighting and activity.

Gap closure. The primary statistic is the fraction of the default-to-oracle gap closed on each held-out real site, with bootstrap confidence intervals over subjects. Synthetic-to-real validity is the central check. Rules are fitted on synthetic sites alone, and we measure whether their gap closure on real MMPD reaches at least half of the gap closure they achieve on held-out synthetic sites. Falling well short means the simulator does not capture real phone acquisition.

Fingerprint distance. We report the rank correlation between the distance $d$ and per-site, and per-clip, error. We also report the error of predictions retained when the highest-$d$ clips are abstained on.

Rolling shutter. We report waveform correlation with the oximeter and inter-region phase consistency, with and without the row-aware pairing rule, on the subset with measurable skew.

Decisive outcomes. The proposal succeeds if three things hold: the configurator beats augmentation-only on MMPD with augmentation in both arms, it closes a majority of the oracle gap on at least two real held-out sites, and the distance $d$ predicts failure. It fails if augmentation-only matches it, regardless of synthetic-grid results.