Emotion-recognition models trained on heart rate, skin conductance and breathing are usually presented as reading a person's inner state from their body. In the standard benchmarks, though, every participant watches the same small set of clips, and the emotion labels track which clip was shown. A model can therefore score well by recognizing the clip from the body's shared, stimulus-locked response and then outputting that clip's typical rating. That score says nothing about how this particular person appraised it. We propose a set of controlled ablations that splits reported accuracy into two parts: a stimulus-prior part, which a model could get from clip identity alone, and a person-specific bodily part. The person-specific part is the one interoceptive theories of emotion actually predict. Interoception here means the brain's sensing of internal bodily signals such as heartbeat, skin conductance, respiration and temperature.
This matters beyond benchmark hygiene. The aesthetic-chills literature shows that the emotionally interesting signal lives between people on the same stimulus. On identical clips, people who get chills report higher valence and arousal than those who do not (Aesthetic chills cause an emotional drift in valence and arousal). Active-inference accounts treat felt valence as a function of how a person's own bodily prediction errors evolve over time (Tschantz et al.). If physiological recognizers only capture what a clip does to everyone, they cannot test or serve either idea. The borrowed tool is the "blindfold" baseline from component contribution analysis, the study of which parts of a model or pipeline actually produce its results. Blindfold Baselines for Embodied QA removed the visual stream and showed that a question-only bag-of-words model reached 50.34% on EQAv1, beating navigation agents at longer distances. The clip-identity prior is the affective analogue of that question-only model, and as far as the file shows, no one has run it.
Core Claim
We hypothesize that a large share of reported cross-subject accuracy on DEAP and AMIGOS is recoverable from clip identity alone. We further hypothesize that the genuinely interoceptive part, meaning a person's deviation from the clip's typical rating predicted from that person's deviation from the clip's typical bodily response, is small but nonzero. We expect that part to be carried by visceral channels (cardiac, electrodermal, respiratory, thermal) rather than by somatomotor channels (eye movements, muscle activity).
The channel prediction is sharpened by an existing result. Zhou et al.'s SHAP-explained tree ensembles reached peripheral-only leave-one-subject-out F1 of 0.814 (valence) and 0.823 (arousal) on DEAP. Their most important features were vertical eye movements, respiration and trapezius muscle activity. Two of those three are not visceral at all, and eye movements are plausibly locked to what is on screen. We read their result as motivating evidence that strong physiological scores may be stimulus recognition, not as a contradiction. Our experiments test that reading directly.
Formally, write each rating as $y_{s,c} = \mu_c + r_{s,c}$, where $y_{s,c}$ is subject $s$'s rating of clip $c$, $\mu_c$ is the mean rating of clip $c$ over training subjects only, and $r_{s,c}$ is the person-specific residual. We define the interoceptive share of a model as
$$\text{IS} = \frac{\text{Perf}(\text{full model}) - \text{Perf}(\text{clip prior})}{\text{Perf}(\text{full model}) - \text{Perf}(\text{chance})},$$
where Perf is the benchmark's own metric computed under the same split. IS near zero means the model adds nothing beyond knowing which clip was shown. We report IS alongside a direct score on predicting $r_{s,c}$.
Research Program
We re-implement three physiology-driven recognizers and run all of them under one leave-one-subject-out protocol on DEAP, AMIGOS and MPED.
The first is the LightGBM pipeline of Zhou et al., which decomposes each signal with singular spectrum analysis (splitting it into trend and oscillatory components) and summarizes the components with entropy and energy features. The second is a cross-attention model in the style of MVP, in which physiological embeddings query facial Action Unit features. The third is a plain 1D-CNN on physiology alone.
Each model is then subjected to five manipulations.
- Clip-prior blindfold. We replace the model with $\hat{y}_{s,c} = \mu_c$. The clip means are computed on training subjects only, so no information about the test subject leaks in.
- Residual target. We retrain every model to predict $r_{s,c}$ instead of $y_{s,c}$. This forces it to explain individual differences on a fixed stimulus.
- Body swap. At test time, we feed the model another held-out subject's physiology recorded during the same clip, while scoring against the original subject's label. A model that reads stimulus-locked signals will barely notice. A model that reads the person's own body will degrade toward the clip prior.
- Within-clip time shuffle. We permute physiological windows within each clip. This keeps each signal's overall distribution but destroys its temporal trajectory, which is the quantity that the active-inference definition of valence depends on.
- Channel-group coalition ablation. We define five channel groups: cardiac (ECG/PPG), electrodermal, respiratory, skin temperature, and somatomotor (EOG/EMG). Each group is mean-ablated, meaning its values are replaced by their training-set average so the input stays in a realistic range. We retrain on all 32 group subsets and compute exact group-level Shapley values, which average each group's marginal contribution over every combination of the other groups.
We use mean ablation rather than zeroing for the reason given in Interpretability in the Wild. Zeroed inputs push a model off its usual input range, so a performance drop may reflect brittleness rather than genuine reliance on that input. We use full coalitions rather than single-group knockouts because the same work found backup components that take over when a primary one is removed. Physiological channels are strongly correlated, so similar compensation is likely.
The active-inference framing contributes one constructive element. Following Tschantz et al., we define a person's bodily prediction error as their physiological trajectory minus the average trajectory of training subjects on the same clip. We also compute the rate of change of this quantity. We test whether these residual-physiology features predict residual ratings better than raw physiology does. This is a light, falsifiable operationalization of the theory, not a full generative model.
We have to handle MPED with care. If its seven-class labels are assigned per clip rather than by each participant's self-report, the residual is undefined by construction. In that case we report the task as pure clip recognition and stop there, rather than forcing the analysis.
Baselines
The naive reference point is the chance-level predictor for each benchmark, meaning the majority class or the training-set label mean. It anchors the denominator of IS.
The central baseline is the clip-prior blindfold described above. It is the direct analogue of the question-only model of Blindfold Baselines for Embodied QA. It is the most dangerous competitor because it needs no physiology at all and exploits exactly the structure the benchmarks share. For fairness, it uses the same leave-one-subject-out splits and the same metric as the models it is compared against, and it is estimated only from training subjects. No reported result was found on Lacuna for a clip-identity baseline on DEAP, AMIGOS or MPED. Measuring it is part of the contribution.
The strongest existing physiological method is Zhou et al.'s LightGBM, which reports leave-one-subject-out F1 of 0.814 for valence, 0.823 for arousal and 0.860 for liking on DEAP (Zhou et al.). The strongest deep multimodal comparison points are MVP's reported weighted F1 of 66 ± 4 for DEAP arousal and 58 ± 6 for AMIGOS valence (MVP). AMIGOS's own baselines report mean F1 of 0.56–0.59 for valence and 0.57–0.59 for arousal (AMIGOS). On MPED, the reported figures are approximately 42% for the attention LSTM versus approximately 31% for an SVM (MPED).
We first reproduce each published pipeline on its published split, reporting our replication number next to the cited one. We then move all pipelines to a common split with an equal hyperparameter search budget of 50 trials each, tuned on inner folds of training subjects only. The clip prior receives the same treatment.
A physiological model "beats" the stimulus prior only under two conditions. First, its F1 must exceed the clip prior's with a 95% subject-bootstrap interval that excludes zero. Second, it must lose a significant part of that advantage under body swap.
The key ablation that isolates our proposed ingredient is the comparison between residual-physiology features and raw physiology, both trained on the residual target.
Why not just…?
Why not just run standard unimodal-versus-multimodal ablations? AMIGOS already did something close and found that fusion did not beat single modalities. Removing a modality tells you whether that modality adds accuracy. It does not tell you whether the accuracy reflects the person's appraisal or recognition of the clip. Every modality, including skin conductance, can carry clip identity. Only the clip prior and body swap separate these two readings, so modality ablation appears in our design but cannot answer the question alone.
Why not just read SHAP attributions, as Zhou et al. did? SHAP explains which features a trained model relies on, not what those features encode. Zhou et al.'s top features being eye movements and trapezius muscle activity is exactly the ambiguity at issue: stimulus-locked behavior can look like "important physiology." Our coalition ablation measures reliance, and the residual target then asks whether that reliance survives once the stimulus is controlled for. The residual target is what SHAP cannot provide.
Why not just evaluate leave-one-clip-out? This is a real threat to our design, because holding out clips also removes clip identity as a shortcut. We include it as a condition. It confounds two effects, however: generalizing to unseen stimuli and ignoring stimulus priors. With only 40 DEAP clips, a drop under leave-one-clip-out could reflect either. If leave-one-clip-out and the residual target give the same verdict, the simpler protocol suffices and we will say so. If they disagree, body swap arbitrates.
Why not just use the clip mean and stop? If the clip prior matches every physiological model, that is itself the headline result. It would mean current benchmarks cannot test interoceptive appraisal, which would argue for datasets built around within-stimulus variation, such as physiology recorded during ChillsDB clips (ChillsDB).
Decisive Checks
Three outcomes settle the question.
The first is IS on DEAP and AMIGOS for each pipeline. We predict IS below 0.5 for valence. A value near 1 would refute our core concern.
The second is the body-swap drop. A model reading individual bodies should fall most of the way to the clip prior. No drop means its gains were stimulus-locked.
The third is the sign and size of the group Shapley values on the residual target. We predict that the visceral groups (cardiac, electrodermal, respiratory, temperature) contribute positively to residual prediction, while the somatomotor group's contribution largely vanishes once the target is residualized. If the somatomotor group still dominates residual prediction, that weakens the interoceptive reading. It would suggest the benchmarks' individual differences are expressed through behavior rather than the viscera.
We also report whether residual-physiology features beat raw features on the residual target. A positive result would be modest empirical support for the prediction-error view of appraisal from Tschantz et al.. A null result would bound how much that view can currently be tested with passive-viewing data.
References
- Aesthetic chills cause an emotional drift in valence and arousal
- Simulating homeostatic, allostatic and goal-directed forms of interoceptive control using active inference
- Blindfold Baselines for Embodied QA
- Cross-Subject Emotion Recognition with Sparsely-Labeled Peripheral Physiological Data Using SHAP-Explained Tree Ensembles
- MVP: Multimodal Emotion Recognition Based on Video and Physiological Signals
- Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small
- AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups
- MPED: A Multi-Modal Physiological Emotion Database for Discrete Emotion Recognition
- ChillsDB: A Gold Standard for Aesthetic Chills Stimuli