FMNet: Latent Feature-wise Mapping Network for Cleaning up Noisy Micro-Doppler Spectrogram

Chong Tang, Wenda Li, Shelly Vishwakarma, Fangzhan Shi, Simon Julier, Kevin Chetty

Abstract

Micro-Doppler signatures contain considerable information about target dynamics. However, radar sensing systems are easily affected by noisy surroundings, resulting in uninterpretable motion patterns on the micro-Doppler spectrogram. Meanwhile, radar returns often suffer from multipath, clutter and interference. These issues lead to difficulty in motion feature extraction and activity classification. In this paper, we propose a latent feature-wise mapping strategy, called Feature Mapping Network (FMNet), to transform measured spectrograms so that they more closely resemble the output from a simulation under the same conditions. Based on measured spectrograms and matched simulated data, our framework contains three parts: an Encoder to extract latent features, a Decoder to reconstruct spectrograms, and a Discriminator to minimize the distance between latent features of measured and simulated data. We demonstrate FMNet with six activities and two experimental scenarios, showing strong enhanced patterns while preserving actual motion information. Furthermore, we propose training a classifier with only simulated data and predicting new measured samples after cleaning them with FMNet, resulting in significant classification improvements.

Introduction: The Reality Gap in Radar Sensing

Micro-Doppler signatures (\mu-DS) are "motion fingerprints" captured by radar, revealing how parts of a target move over time. They are invaluable for activity recognition, but real-world indoor environments introduce significant noise, multipath reflections, and clutter.

Traditional denoising approaches typically require pairs of noisy and clean data. However, in radar sensing, capturing a "clean" version of a real-world measurement is nearly impossible. While synthetic noise can be added to simulations, it rarely captures the complex interference found in actual environments. FMNet addresses this "reality gap" by focusing on feature alignment in a latent space rather than simple pixel-to-pixel mapping.

Methods: The Feature Mapping Network (FMNet)

The FMNet architecture combines Variational Autoencoders (VAE) and Adversarial Autoencoders (AAE) to align the internal representations of noisy real-world signals with high-fidelity simulations.

Detailed FMNet architecture and the three-phase training strategy.

The training process involves three distinct phases:

  1. Reconstruction & Regularization: The network is trained on simulated data to learn a well-structured latent space representing clean motion.
  2. Feature Mapping: Using an adversarial approach, the network learns to map the latent features of measured (noisy) data to overlap with those of simulated (clean) data. The Discriminator ensures the features are indistinguishable.
  3. Content Consistency: A final refinement step uses Kullback-Leibler Divergence (KLD) to ensure the enhanced output preserves the original motion information, preventing the generation of arbitrary "clean-looking" images.

Datasets and Experimental Results

The study utilized measured data from a Passive WiFi Radar (PWR) system and simulated data generated via SimHumalator, which converts Kinect motion capture data into realistic radar signatures.

Comparison between simulated and measured micro-Doppler spectrograms for various activities.

FMNet significantly outperformed standard mapping networks, producing sharper spectrograms with fewer artifacts. The most striking result was in cross-domain classification: a VGG16 model trained entirely on simulated data achieved 93.9% accuracy on measured samples cleaned by FMNet. In contrast, accuracy on raw, uncleaned measured data was only approximately 58%.

Conclusion

FMNet demonstrates that by "fixing" underlying features rather than pixels, we can bridge the gap between simulation and reality. This allows for the development of complex activity recognition models in perfect virtual environments that remain highly effective when deployed in messy, real-world radar sensing scenarios. Future work will focus on scaling this approach to more diverse environments and multi-person scenarios.

Summary updated

Abstract

Micro-Doppler signatures contain considerable information about target dynamics. However, the radar sensing systems are easily affected by noisy surroundings, resulting in uninterpretable motion patterns on the micro-Doppler spectrogram. Meanwhile, radar returns often suffer from multipath, clutter and interference. These issues lead to difficulty in, for example motion feature extraction, activity classification using micro Doppler signatures ($\mu$-DS), etc. In this paper, we propose a latent feature-wise mapping strategy, called Feature Mapping Network (FMNet), to transform measured spectrograms so that they more closely resemble the output from a simulation under the same conditions. Based on measured spectrogram and the matched simulated data, our framework contains three parts: an Encoder which is used to extract latent representations/features, a Decoder outputs reconstructed spectrogram according to the latent features, and a Discriminator minimizes the distance of latent features of measured and simulated data. We demonstrate the FMNet with six activities data and two experimental scenarios, and final results show strong enhanced patterns and can keep actual motion information to the greatest extent. On the other hand, we also propose a novel idea which trains a classifier with only simulated data and predicts new measured samples after cleaning them up with the FMNet. From final classification results, we can see significant improvements.
BibTeX
@article{tang2022fmnet,
  author = {Chong Tang and Wenda Li and Shelly Vishwakarma and Fangzhan Shi and Simon Julier and Kevin Chetty},
  title = {{FMNet: Latent Feature-wise Mapping Network for Cleaning up Noisy Micro-Doppler Spectrogram}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2022},
  url = {https://openreview.net/forum?id=L4AT9uYqpV},
}
Figure