Abstract
This work presents an application of Integrated sensing and communication (ISAC) system for monitoring human activities directly related to healthcare. Real-time monitoring of humans can assist professionals in providing healthy living enabling technologies to ensure the health, safety, and well-being of people of all age groups. To enhance the human activity recognition performance of the ISAC system, we propose to use synthetic data generated through our human micro-Doppler simulator, SimHumalator to augment our limited measurement data. We generate a more realistic micro-Doppler signature dataset using a style-transfer neural network. The proposed network extracts environmental effects such as noise, multipath, and occlusions effects directly from the measurement data and transfers these features to our clean simulated signatures. This results in more realistic-looking signatures qualitatively and quantitatively. We use these enhanced signatures to augment our measurement data and observe an improvement in the classification performance by 5% compared to no augmentation case. Further, we benchmark the data augmentation performance of the style transferred signatures with three other synthetic datasets -- clean simulated spectrograms (no environmental effects), simulated data with added AWGN noise, and simulated data with GAN generated noise. The results indicate that style transferred simulated signatures well captures environmental factors more than any other synthetic dataset.
Introduction
Deep learning has significantly advanced Human Activity Recognition (HAR) using Integrated Sensing and Communication (ISAC) systems. By utilizing WiFi signals as a form of radar, these systems can monitor activities without invasive cameras. However, training robust deep learning models requires vast amounts of labeled data. Collecting real-world radar data for critical activities, such as falls, is logistically challenging and potentially hazardous.
While simulators like SimHumalator can generate large volumes of synthetic data, these micro-Doppler signatures are often "too perfect." They lack the complex environmental noise, multipath interference, and signal occlusions characteristic of real-world indoor settings. Consequently, models trained solely on clean simulations often fail to generalize to real-world deployments. This research addresses the "reality gap" by proposing a Neural Style Transfer (NST) framework to synthesize realistic environmental effects onto clean simulated data.
Methodology
The proposed framework treats the kinematic motion of a person as "content" and the environmental interference as "style." By applying NST to radar spectrograms, the researchers can bridge the gap between simulation and reality without the need for computationally intensive electromagnetic physics modeling.
The core of the pipeline utilizes a pre-trained VGG-19 network to extract feature representations. The optimization process minimizes a joint loss function consisting of:
- Content Loss: Ensures the underlying human activity (e.g., walking, sitting) remains recognizable in the spectrogram.
- Style Loss: Calculated using Gram matrices to capture the statistical "texture" of environmental noise and multipath effects from a small set of real-world measurements.
Experimental Setup and Datasets
The study utilized two primary data sources:
- Measured Data: 600 samples collected from 5 participants performing 10 activities (e.g., falling, punching, sitting) using a passive ISAC system comprising a WiFi AP and USRP-2921 receivers.
- Simulated Data: 600 clean samples generated via SimHumalator.
To evaluate the effectiveness of NST, the style-transferred data was benchmarked against clean simulations, simulations with Additive White Gaussian Noise (AWGN), and simulations with GAN-generated noise.
Results and Discussion
The application of NST successfully synthesized realistic environmental artifacts onto the simulated signatures. Qualitatively, the style-transferred images closely resemble real-world measurements, capturing the "messiness" of the signal while preserving the kinematic structure of the activity.
Key findings include:
- Accuracy Improvement: Augmenting the training set with style-transferred signatures resulted in a 5% increase in classification accuracy compared to using real data alone.
- Feature Space Alignment: t-SNE visualizations and Euclidean distance metrics confirmed that NST-generated data occupies a similar latent feature space to real measurements (Mean distance of 5.9), significantly outperforming AWGN-augmented data (Mean distance of 25.7).
- Data Scarcity Mitigation: In scenarios where 80% of real data was replaced by synthetic data, the NST-enhanced model maintained performance levels nearly identical to the full real-world dataset, demonstrating its value for "cold-start" sensing applications.
Conclusion
This work demonstrates that treating environmental noise as a "style" allows for the creation of high-fidelity synthetic training data for HAR. By preserving kinematic fidelity while adding realistic interference, NST provides a scalable and effective method for training robust healthcare monitoring systems. Future work will explore faster, single-pass NST models and investigate the generalization of style transfer across diverse architectural environments and different data types like Channel State Information (CSI).
Summary updated