FMNet: Latent Feature-Wise Mapping Network for Cleaning Up Noisy Micro-Doppler Spectrogram

Chong Tang, Wenda Li, Shelly Vishwakarma, Fangzhan Shi, Simon J. Julier, Kevin Chetty

TL;DR

FMNet is a deep learning framework designed to "clean" noisy micro-Doppler radar signatures by mapping real-world measured data into a clean, simulated latent space. By aligning the internal representations of messy real-world signals with high-fidelity simulations, the researchers achieved a 93.9% classification accuracy using a model trained entirely on simulated data.

What Problem Does This Paper Address?

Micro-Doppler signatures ($\mu$-DS) are like "motion fingerprints" captured by radar, revealing how parts of a target (like a person's arms or legs) move over time. They are incredibly useful for activity recognition, but in the real world—especially indoors—they are plagued by noise, multipath reflections (signals bouncing off walls), and clutter.

The traditional "denoising" approach requires pairs of noisy and clean data to train a neural network. However, in radar sensing, it is nearly impossible to capture a "clean" version of a real-world noisy measurement. While researchers often add artificial noise to simulations to create training pairs, this synthetic noise rarely matches the complex, spatially correlated interference found in actual environments. FMNet tackles this "reality gap" by focusing on feature alignment rather than simple pixel-to-pixel mapping.

Methods

The authors propose the Feature Mapping Network (FMNet), which combines the strengths of Variational Autoencoders (VAE) and Adversarial Autoencoders (AAE). The architecture consists of an Encoder (extracts features), a Decoder (reconstructs the image), and a Discriminator (aligns distributions).

The training happens in three distinct phases:

  1. Reconstruction & Regularization: The network is trained on simulated data to learn a well-structured "latent space" (a compressed mathematical representation of clean motion).
  2. Feature Mapping: Using an adversarial approach, the network learns to map the latent features of measured (noisy) data so they overlap with the latent features of simulated (clean) data. The Discriminator's job is to ensure it can't tell the difference between a "real" feature and a "simulated" one.
  3. Content Consistency Check: A final refinement step uses Kullback-Leibler Divergence (KLD) to ensure that the enhanced output preserves the actual motion information from the original input, preventing the network from just generating a random "clean-looking" image.

Datasets

  • Measured Data: Collected using a Passive WiFi Radar (PWR) system. Three participants performed six activities (e.g., sit-down, stand-up, walk-to-fall).
  • Simulated Data: Generated using SimHumalator, an open-source framework that converts Kinect motion capture (Mocap) data into realistic WiFi radar signatures.
  • Scenarios: The team tested both Line-of-Sight (LOS) and Through-the-Wall (TTW) scenarios to evaluate the model's robustness against extreme signal attenuation.

Key Insights and Ideas

  • Latent Space Alignment: Instead of trying to "fix" pixels, FMNet "fixes" the underlying features. If the network's internal representation of a "noisy walk" looks like its representation of a "clean walk," the decoder will naturally produce a clean output.
  • Bypassing the Label Problem: By using simulations as the "gold standard" for features, the researchers don't need perfectly clean real-world labels, which are impossible to get.
  • Simulation-to-Reality Transfer: The most striking insight is that once FMNet cleans the data, a classifier trained only on simulated data can accurately identify real-world activities. This suggests we can train complex models in "perfect" virtual environments and deploy them in the "messy" real world.

Key Results

  • Visual Clarity: FMNet significantly outperformed standard mapping networks (SMNet), producing sharper spectrograms with fewer artifacts, especially in the challenging Through-the-Wall (TTW) tests.
  • Quantitative Superiority: FMNet achieved higher Structural Similarity Index (SSIM) scores and lower pixel loss compared to non-regularized versions.
  • Classification Boost: In a cross-domain test, a VGG16 classifier trained on simulated data achieved 93.9% accuracy on measured samples cleaned by FMNet. Without FMNet cleaning, the accuracy on raw measured data was significantly lower (around 58%).

Limitations and Future Work

  • Dataset Size: The study used a relatively small dataset (304 pairs). Scaling this to more diverse environments and a larger variety of human body types would be a logical next step.
  • Complexity of Scenes: The current work focuses on single-person activities in controlled rooms. Future work could explore multi-person scenarios or more complex "cluttered" environments like factories or crowded public spaces.
  • Real-time Processing: While the paper demonstrates the effectiveness of the architecture, it doesn't deeply explore the computational latency required for real-time "cleaning" in edge-deployed radar systems.

Summary updated

Abstract

Micro-Doppler signatures contain considerable information about target dynamics. However, the radar sensing systems are easily affected by noisy surroundings, resulting in uninterpretable motion patterns on the micro-Doppler spectrogram ( $\mu $ -DS). Meanwhile, radar returns often suffer from multipath, clutter, and interference. These issues lead to difficulty in, for example, motion feature extraction and activity classification using micro-Doppler signatures. In this article, we propose a latent feature-wise mapping strategy, called feature mapping network (FMNet), to transform measured spectrograms so that they more closely resemble the output from a simulation under the same conditions. Based on measured spectrogram and the matched simulated data, our framework contains three parts: an encoder which is used to extract latent representations/features, a decoder outputs reconstructed spectrogram according to the latent features, and a discriminator minimizes the distance of latent features of measured and simulated data. We demonstrate the FMNet with six activities data and two experimental scenarios, and final results show strong enhanced patterns and can keep actual motion information to the greatest extent. On the other hand, we also propose a novel idea which trains a classifier with only simulated data and predicts new measured samples after cleaning them up with the FMNet. From final classification results, we can see significant improvements.
BibTeX
@article{tang2022fmnet,
  author = {Chong Tang and Wenda Li and Shelly Vishwakarma and Fangzhan Shi and Simon J. Julier and Kevin Chetty},
  title = {{FMNet: Latent Feature-Wise Mapping Network for Cleaning Up Noisy Micro-Doppler Spectrogram}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2022},
  url = {https://openreview.net/forum?id=L4AT9uYqpV},
}

Loading related content...

Figure