Bio-PM-Guided Synthetic IMU Generation

Deepali Singh, Deepesh Parpani, Aviral GuptaarXiv 2026 · arXiv

TL;DR

Researchers at UMass Amherst have developed a way to generate high-quality synthetic motion sensor data by working in the "latent space" of a pretrained biosignal encoder. By using a class-conditional diffusion model (CD-IMU) to generate these abstract representations, they significantly improved the accuracy of Human Activity Recognition (HAR) models on new, unseen subjects, particularly for challenging postural activities.

What Problem Does This Paper Address?

Human Activity Recognition (HAR)—the tech that lets your smartwatch know if you're walking, jogging, or sitting—relies heavily on Inertial Measurement Unit (IMU) data. However, collecting and labeling this data is notoriously expensive and time-consuming.

Even worse, HAR models often suffer from a "generalization gap": a model trained on one group of people frequently fails when tested on a new individual because everyone moves slightly differently. While synthetic data augmentation is a common fix, generating realistic raw IMU signals is incredibly difficult due to their complex temporal patterns and subject-specific "noise." This paper asks: what if we don't try to generate the raw signal at all, but instead generate the meaningful features of that signal?

Methods

The authors leverage Bio-PM, a large-scale Transformer encoder pretrained on 28,000 hours of wrist-IMU data. Bio-PM acts as a "quality oracle," converting raw 10-second motion windows into a sequence of 192 token embeddings. These tokens already encode the semantic structure of the movement.

The researchers developed and compared two generative decoders to create synthetic versions of these tokens:

  1. CAR-IMU (Autoregressive): A Transformer that predicts tokens one by one. While efficient, it suffered from "signal fading" (forgetting the activity class halfway through the sequence) and "drift" (compounding errors over time).
  2. CD-IMU (Diffusion): A class-conditional diffusion model that learns the entire token distribution through a denoising objective. It uses FiLM (Feature-wise Linear Modulation) to inject the activity class information at every single layer, ensuring the model never "forgets" what it's supposed to be generating.

To prove these synthetic tokens are useful, they also built a PatchDecoder—a small MLP that can approximately reconstruct raw waveforms from the generated tokens, allowing for visual inspection of the synthetic "movement."

Datasets

  • WISDM: A benchmark wrist accelerometer dataset featuring 51 subjects performing six activities (Walking, Jogging, Upstairs, Downstairs, Sitting, Standing).
  • NHANES: The massive corpus (11,000 participants) used to pretrain the underlying Bio-PM encoder.

Key Insights and Ideas

  • Latent Space is the Place: Generating data in a pretrained representation space is much more effective than generating raw signals. As shown in Figure 1, the Bio-PM token space naturally clusters activities without being biased by individual subject styles, making it a perfect target for class-conditional generation.
Fig. 1: UMAP activity clusters on real data tokens generated by Bio-PM
  • Diffusion Beats Autoregression: The diffusion-based CD-IMU outperformed the autoregressive model because it models the global context of the window simultaneously. Figure 2 (specifically panels a and b) shows how synthetic tokens (red crosses) are perfectly intermixed with real tokens (blue circles), proving the model has captured the true distribution of the data.
Fig. 2: Embedding space comparison of real and CD-IMU synthetic tokens at two levels of granularity. (a, b) Raw individual 64-d token embeddings sampled across all sequence positions: real (blue circles) and synthetic (red crosses) are intermixed throughout all clusters, confirming distributional alignment at the patch level. (c, d) Mean-pooled window-level features colored by activity class: synthetic tokens for each class co-locate with their real counterparts, preserving the semantic structure of Bio-PM’s token space. Together, (a–d) demonstrate that CD-IMU generates tokens that are realistic both at the individual patch position level and at the window-level activity representation level.
  • Confusion-Aware Augmentation: Instead of just adding 5% more data across the board, the authors used "difficulty-aware" augmentation. They added more synthetic data for classes where the baseline model was most confused (like "Sitting" vs. "Standing"), leading to much larger performance gains.

Key Results

The results demonstrate that synthetic tokens are not just "realistic-looking" but practically useful:

  • TSTR Success: The CD-IMU achieved a Train-on-Synthetic-Test-on-Real (TSTR) macro-F1 of 0.68, meaning a classifier trained only on fake data can still recognize real human activities with high accuracy.
  • Performance Boost: Adding CD-IMU synthetic tokens to a real-only baseline raised the Leave-One-Subject-Out (LOSO) macro-F1 from 0.729 to 0.809.
  • Postural Breakthrough: The "Sitting" activity saw a massive improvement of +0.191 F1 points, largely thanks to the inclusion of gravity-aware embeddings that help distinguish static postures.
Fig. 3: Per-class and macro-average F1 for Baseline, CD-Exp1, and CD-Exp2 across 51 LOSO folds on WISDM. CD-Exp2 consistently improves F1 across all activity classes.

Limitations and Future Work

While the token generation is high-fidelity, the PatchDecoder for raw signal reconstruction has some hurdles. Because it decodes each token independently, the resulting waveforms can have "seams" or discontinuities at the boundaries between patches. Additionally, the reconstructed signals often show amplitude attenuation—the "peaks" of a jogging motion aren't as sharp as in real data.

The authors suggest that future work should focus on context-aware reconstruction and "overlap-add" stitching techniques to create smoother, more dynamic raw signal outputs.

Summary updated

Abstract

Collecting and labeling inertial measurement unit (IMU) data for human activity recognition (HAR) is expensive, and models trained on one group of subjects often fail to generalize to unseen individuals. Even with balanced data, classifiers struggle to distinguish dynamically similar activity classes and benefit from targeted augmentation. We explore whether synthetic IMU data generated in the representation space of a pretrained biosignal encoder, Bio-PM \cite{b1}, can serve as a practical augmentation source. Rather than synthesizing raw signals, we leverage Bio-PM's frozen encoder to convert 10-second IMU windows into token embedding sequences and train class-conditional decoders in this space: (1) an autoregressive transformer decoder that predicts tokens sequentially, and (2) a diffusion decoder that learns the token distribution through a denoising objective. We analyze generation realism through distributional metrics and embedding-space visualizations, evaluate downstream utility through leave-one-subject-out HAR classification, and present a PatchDecoder for approximate raw waveform reconstruction. Our diffusion decoder achieves a Train-on-Synthetic-Test-on-Real (TSTR) macro-F1 of 0.68, demonstrating that generated tokens are realistic enough to train a classifier that transfers to real data. Combining synthetic-token augmentation with a 320-dimensional gravity-aware feature representation raises LOSO macro-F1 from 0.729 to 0.809 over a real-only baseline across 51 subjects on the WISDM benchmark, with statistically significant gains concentrated in the most challenging activity classes. These results demonstrate that high-fidelity generation in a pretrained encoder's representation space is achievable and practically useful for cross-subject HAR augmentation.
BibTeX
@misc{singh2026bio,
  author = {Deepali Singh and Deepesh Parpani and Aviral Gupta},
  title = {{Bio-PM-Guided Synthetic IMU Generation}},
  year = {2026},
  eprint = {2611.26607},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26607},
}
Figure