Mode-Anchored Test-Time Normalization for Wearers Who Mostly Sit Still

Activity-recognition models for wrist and body sensors are trained mostly on healthy young adults. When they are deployed on an older, impaired, or institutionalized person, they fail in a predictable way. Consumer wristbands worn by nursing-home residents with dementia labeled most of the day as "Inactive" or "Sleeping," even though the raw accelerometer signal tracked clinical observation well (Using consumer wearables to estimate physical activity of nursing home residents with dementia). The standard fix for a shifted test population is test-time normalization. It recomputes, on the unlabeled test stream, the per-channel averages and spreads that a network uses to rescale its internal activations. That fix assumes the test stream contains roughly the same mix of activities as training. A person who sits still 90% of the time breaks this assumption. The test-stream average then tracks the resting activity rather than the person's sensor offset, and normalization erases exactly the differences the classifier needs. On images, this failure is dramatic: recomputing statistics on single-class test batches cost 73.9 accuracy points on CIFAR-10-C (Channel Selection for Test-Time Adaptation Under Distribution Shift).

We propose to turn the skew into an anchor. When one activity dominates the stream, the densest point of each channel's activation distribution, its mode, is a clean, outlier-resistant estimate of where that activity sits for this wearer. The method stores, from training, where each activity's activations peak in each channel. That is a few numbers per channel per class, not source data. At test time, it finds the wearer's dominant activation peak, identifies which activity that peak belongs to, and shifts the channel so the two peaks coincide. In two sentences: instead of matching the test stream's average to the training average, which is wrong whenever the activity mix changes, match the test stream's highest-density peak to the training peak of the same activity. Channels are adapted only where that peak is sharp and its activity identity is unambiguous.

Core Claim

The claim is that per-subject offset can be estimated robustly under extreme label skew by matching class-conditional activation modes, and that this beats mean-based test-time normalization exactly in the regime vulnerable cohorts live in. The rest of this section defines the correction and then places it against prior work.

For channel $c$, let $\hat m_{T,c}$ be the mode of that channel's activations over a short buffer of recent test windows. It is found by a few fixed-point mean-shift iterations, a hill-climb that repeatedly moves an estimate to the kernel-weighted average of nearby points. Let $k^\*$ be the activity the frozen network predicts for the windows inside that mode's basin of attraction. Let $m_{S,c,k}$ be the stored training mode of channel $c$ for activity $k$. The correction is

$$\tilde a_c = a_c - \lambda_c\,\big(\hat m_{T,c} - m_{S,c,k^\*}\big),$$

where $a_c$ is the incoming activation and $\lambda_c \in [0,1]$ is a per-channel gate. The gate is set to zero unless the target peak is sharp and the basin's predicted activity is consistent. When the buffer holds several clear peaks, each is matched to its own activity and the offsets are averaged by mass. This generalizes the single-peak case.

The kernel bandwidth used in the mode search is fixed from the training spread of each activity's activations in that channel. This sidesteps the bandwidth sensitivity that plagues kernel modal methods. A bandwidth chosen this way is matched to the width of a real class cluster, so it is neither so small that it chases noise nor so large that the mode collapses toward the mean.

Prior work divides into three roles. Two papers motivate the problem. Channel Selection for Test-Time Adaptation Under Distribution Shift (Hybrid-TTN) shows that label shift breaks normalization and that adapting only some channels helps. It chooses channels by a Wasserstein distance between Gaussian summaries, means and variances, and it is evaluated only on corrupted images. The dementia wearable study documents the extreme sedentary skew of the target population.

Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition (OFTTA) is the closest method on our data. It blends source and target means and variances, adapting shallow layers more than deep ones, and adds a prototype classifier, all without backpropagation. It never conditions its statistics on activity identity. It therefore inherits the assumption that the test mix resembles training.

Two lines supply the method. The fixed-point mode search with a monotone ascent property comes from Robust modal regression with direct gradient approximation of modal regression risk. The argument that density peaks resist contamination where means do not comes from Modal Principal Component Analysis. That paper reports a subspace error of 1.4 against 67.0 for classical PCA on contaminated Thyroid data.

One paper partly overlaps. Discretized Density-Guided Source-Free Domain Adaptation for Regression (MERCI) also uses density peaks during adaptation. It places them on the output of a regression model, assumes a single global peak, and relies on Monte Carlo dropout and global density estimation. That is too heavy for a wrist device. We place peaks on internal channels and condition them on activity, and the method needs no gradient steps.

Research Program

The work has three parts.

The first part is the estimator and its analysis. We will characterize when the mode-matching offset is unbiased. Under a pure per-subject shift plus an arbitrary activity mix, the class-conditional mode shifts by exactly the offset, whatever the mix is. The mean does not have this property. We will then quantify how bias grows when the shift also changes cluster shape, for example when a frail walker's "walking" cluster is both displaced and narrower. This analysis tells us when scale should also be corrected. If so, the ratio of peak half-widths between the matched source and target clusters is the natural candidate. Peak half-width is the distance from the mode at which density falls to half its maximum.

The second part is the gate, which is where feature redundancy enters. Wearable networks fed by many sensor axes have many correlated channels. Hybrid-TTN shows that adapting all of them is harmful under label shift. We will compare three gating rules. The first is Hybrid-TTN's Wasserstein criterion. The second is a mode-sharpness criterion: the peak density relative to its bandwidth, together with basin-label agreement. The third pools offset estimates across groups of highly correlated channels before gating. Pooling uses redundancy to cut estimation variance on small buffers, rather than discarding redundant channels as static pruning would (Gaussian Image Anomaly Detection with Greedy Eigencomponent Selection).

The third part is engineering and measurement on edge hardware. The mean-shift search is one-dimensional per channel over a fixed buffer, so its cost is linear in buffer size. We will measure latency on a Raspberry Pi 3B+ against OFTTA's reported 69 ms per instance, and confirm the method stays in the same range.

Baselines

The naive reference is the source model with frozen training statistics. Every adaptation method must beat it, and under heavy skew some may not. It is run on identical test streams with no tuning.

The parameter-free standard is full test-time recomputation of normalization statistics (The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization). We pair it with its small-batch-stable variant, MemBN, which keeps a memory queue of past test statistics. MemBN is given the same buffer length as our method so that the two see identical amounts of target data.

The strongest existing competitor on wearable data is OFTTA. It is the only method in the file designed and validated for cross-person activity recognition under edge constraints. Its paper reports an average accuracy gain of 3.88% under leave-one-subject-out adaptation and 2.77% under continual adaptation across UCI-HAR, OPPORTUNITY and UniMiB-SHAR, but gives no per-dataset numbers. We will therefore rerun it on the same backbones and splits, and tune its decay factor on the same held-out validation subjects with the same search budget as our gate threshold. Beating OFTTA means a higher macro-F1 on skewed streams, with confidence intervals over subjects excluding zero, and no loss larger than one point on naturally balanced streams.

The strongest method for label-shift-robust normalization is Hybrid-TTN. It reports accuracy changes of −7.2 versus −73.9 for standard normalization on CIFAR-10-C, and −11.6 versus −70.3 on ImageNet-1K-C, under one-class label shift. No reported result was found on Lacuna for Hybrid-TTN on any wearable dataset, so we will port it to the same backbones with the same channel budget.

Tent, entropy minimization on normalization parameters, is included as the gradient-based reference. It is reported at 221 ms per instance on the Raspberry Pi.

The ablations isolate the key ingredient. Marginal mode matches the test peak to the overall training peak, so the activity conditioning is removed. Class-conditional mean aligns per-class means estimated from confident pseudo-labels, so the mode is replaced by the mean. Ungated adapts all channels, so channel selection is removed.

Why not just…?

Why not just use the median or a trimmed mean in the normalization layers? Robust location estimators fix contamination by motion artifacts. They do not fix label skew. The median of a 90%-sitting buffer is the sitting cluster's center, exactly as the mode is. Subtracting it as if it were the population center is the same error the mean makes. A median-statistic normalization arm is included, and the skew sweep below should show it tracking the mean-based methods rather than ours. If it does not, the activity-conditioning argument is wrong.

Why not just port Hybrid-TTN, which already solves label-shift collapse? Hybrid-TTN avoids damage by not adapting channels whose statistics moved, but that leaves those channels uncorrected for the wearer's genuine offset. On images, the residual loss was still 7 to 12 points. Our gate keeps the channels that can be corrected safely and corrects them. This is a real competitor. The head-to-head on skewed streams settles it, and if Hybrid-TTN matches us there, the contribution reduces to the gating rule.

Why not just align class-conditional means using pseudo-labels? This is the most serious threat, because it also conditions on activity. Its weakness is that every pseudo-labeled window contributes to the mean, including misclassified ones and artifact bursts. A sitting person mislabeled as "walking" a fraction of the time drags the walking mean, whereas the mode only needs the dominant basin to be labeled correctly. The class-conditional-mean ablation tests this directly. If it equals the full method under artifact injection, we will report that the mode adds nothing beyond conditioning.

Why not just fine-tune on a little labeled data from the target cohort, as ProGait does? ProGait shows that supervised fine-tuning on prosthesis users recovers accuracy. Per-resident labels in nursing homes, however, require trained observers. Fine-tuning will be reported as an upper bound, not a competitor.

Proposed Experiments

We will use leave-one-subject-out splits on UCI-HAR, OPPORTUNITY and UniMiB-SHAR to compare directly with OFTTA. We will also use HAR-PMD, where 120 participants use crutches, walkers and wheelchairs. Its reported user-independent accuracy of 98.79% on balanced data leaves room to expose failures only under skew. Finally, we will use demographic splits of MoVi, training on one sex and one BMI band of its 17–32 range and testing on the other.

The decisive check is a skew sweep. Each test subject's stream is resampled so the dominant resting class makes up between a balanced share and 95%. We then plot macro-F1 against skew for every method. The hypothesis predicts that mean-, median- and OFTTA-style normalization degrade monotonically, while mode anchoring stays flat. A second check injects sensor-displacement bursts and one-sided resting-baseline drift. A third measures per-instance latency on the Raspberry Pi 3B+.

If the dementia wearable recordings can be obtained, we will run a qualitative transfer check on them. That check asks whether the adapted model's estimated time inactive tracks the mean amplitude deviation signal better than the unadapted model does. The study's 0.77 correlation between mean amplitude deviation and the watch's "Light" class gives the reference scale.

The idea is falsified under any of three outcomes: the class-conditional-mean ablation matches the full method across skew and artifacts, Hybrid-TTN matches it on skewed streams, or the gain vanishes at realistic buffer lengths.