TL;DR
Researchers at UMass Amherst have developed a way to generate high-quality synthetic motion sensor data by working in the "latent space" of a pretrained biosignal encoder. By using a class-conditional diffusion model (CD-IMU) to generate these abstract representations, they significantly improved the accuracy of Human Activity Recognition (HAR) models on new, unseen subjects, particularly for challenging postural activities.
What Problem Does This Paper Address?
Human Activity Recognition (HAR)—the tech that lets your smartwatch know if you're walking, jogging, or sitting—relies heavily on Inertial Measurement Unit (IMU) data. However, collecting and labeling this data is notoriously expensive and time-consuming.
Even worse, HAR models often suffer from a "generalization gap": a model trained on one group of people frequently fails when tested on a new individual because everyone moves slightly differently. While synthetic data augmentation is a common fix, generating realistic raw IMU signals is incredibly difficult due to their complex temporal patterns and subject-specific "noise." This paper asks: what if we don't try to generate the raw signal at all, but instead generate the meaningful features of that signal?
Methods
The authors leverage Bio-PM, a large-scale Transformer encoder pretrained on 28,000 hours of wrist-IMU data. Bio-PM acts as a "quality oracle," converting raw 10-second motion windows into a sequence of 192 token embeddings. These tokens already encode the semantic structure of the movement.
The researchers developed and compared two generative decoders to create synthetic versions of these tokens:
- CAR-IMU (Autoregressive): A Transformer that predicts tokens one by one. While efficient, it suffered from "signal fading" (forgetting the activity class halfway through the sequence) and "drift" (compounding errors over time).
- CD-IMU (Diffusion): A class-conditional diffusion model that learns the entire token distribution through a denoising objective. It uses FiLM (Feature-wise Linear Modulation) to inject the activity class information at every single layer, ensuring the model never "forgets" what it's supposed to be generating.
To prove these synthetic tokens are useful, they also built a PatchDecoder—a small MLP that can approximately reconstruct raw waveforms from the generated tokens, allowing for visual inspection of the synthetic "movement."
Datasets
- WISDM: A benchmark wrist accelerometer dataset featuring 51 subjects performing six activities (Walking, Jogging, Upstairs, Downstairs, Sitting, Standing).
- NHANES: The massive corpus (11,000 participants) used to pretrain the underlying Bio-PM encoder.
Key Insights and Ideas
- Latent Space is the Place: Generating data in a pretrained representation space is much more effective than generating raw signals. As shown in Figure 1, the Bio-PM token space naturally clusters activities without being biased by individual subject styles, making it a perfect target for class-conditional generation.
- Diffusion Beats Autoregression: The diffusion-based CD-IMU outperformed the autoregressive model because it models the global context of the window simultaneously. Figure 2 (specifically panels a and b) shows how synthetic tokens (red crosses) are perfectly intermixed with real tokens (blue circles), proving the model has captured the true distribution of the data.
- Confusion-Aware Augmentation: Instead of just adding 5% more data across the board, the authors used "difficulty-aware" augmentation. They added more synthetic data for classes where the baseline model was most confused (like "Sitting" vs. "Standing"), leading to much larger performance gains.
Key Results
The results demonstrate that synthetic tokens are not just "realistic-looking" but practically useful:
- TSTR Success: The CD-IMU achieved a Train-on-Synthetic-Test-on-Real (TSTR) macro-F1 of 0.68, meaning a classifier trained only on fake data can still recognize real human activities with high accuracy.
- Performance Boost: Adding CD-IMU synthetic tokens to a real-only baseline raised the Leave-One-Subject-Out (LOSO) macro-F1 from 0.729 to 0.809.
- Postural Breakthrough: The "Sitting" activity saw a massive improvement of +0.191 F1 points, largely thanks to the inclusion of gravity-aware embeddings that help distinguish static postures.
Limitations and Future Work
While the token generation is high-fidelity, the PatchDecoder for raw signal reconstruction has some hurdles. Because it decodes each token independently, the resulting waveforms can have "seams" or discontinuities at the boundaries between patches. Additionally, the reconstructed signals often show amplitude attenuation—the "peaks" of a jogging motion aren't as sharp as in real data.
The authors suggest that future work should focus on context-aware reconstruction and "overlap-add" stitching techniques to create smoother, more dynamic raw signal outputs.
Summary updated