Dual-Scale Doppler Attention for Human Identification

Sunjae Yoon, Dahyun Kim, Ji Woo Hong, Junyeong Kim, Chang D. Yoo

TL;DR

This paper introduces Dual-Scale Doppler Attention (DSDA), a novel deep learning approach for human identification using micro-Doppler (MD) radar signatures. DSDA effectively captures unique gait characteristics by attending to both fast-moving (limbs) and slow-moving (torso) components within the MD signatures, achieving state-of-the-art performance on the IDRad dataset.

What Problem Does This Paper Address?

Human identification (HI) is crucial for security and surveillance. While visual methods are common, they struggle in low-light conditions and raise privacy concerns. Radar offers a robust alternative, operating in various environments and preserving privacy. Specifically, micro-Doppler (MD) signatures, generated from radar reflections of moving targets, encode unique gait patterns. However, existing methods, often relying on conventional machine learning or basic deep learning architectures, struggle to fully exploit the fine-grained temporal and frequency details within MD signatures, especially when dealing with variations in human movement and environmental conditions. This paper aims to improve human identification accuracy by developing a more sophisticated deep learning model that can better interpret these complex MD signatures.

Methods

The core contribution is the Dual-Scale Doppler Attention (DSDA) model. DSDA leverages a Deep Convolutional Neural Network (DCNN) enhanced with a novel attention mechanism. The key idea is to process the MD signatures at different resolutions to capture both rapid and steady motion components.

Here's a breakdown of the DSDA architecture:

  • Window Generation: The input MD signature is divided into two types of windows:
  • Temporal Windows: These are designed to capture fast-moving features, like those from arms and legs. They are generated by sliding windows across the time dimension of the MD signature.
  • Holistic Windows: These are intended to capture slow-moving features, such as the torso's movement. They are derived from the original MD signature.
  • Window Encoder (WE): Both temporal and holistic windows are passed through a series of convolutional, ELU activation, and max-pooling layers to extract feature representations.
  • Temporal Window Attention (TWA): This component focuses on the fast-moving features. It applies a Window Attention (WA) module to the encoded temporal windows. WA calculates attention weights to highlight important temporal patterns. TWA then outputs attended temporal features and an averaged version that summarizes the overall fast-moving characteristics.
  • Holistic Window Attention (HWA): This component targets slow-moving features. It uses the original holistic features and subtracts the averaged fast-moving features (obtained from TWA) to isolate the slow-moving components. It then applies the WA module to these refined holistic features.
  • Classification: The attended temporal and holistic features are concatenated and fed into a final classifier (a series of Dropout, ELU, and Linear layers) to predict the human identity.

The Window Attention (WA) module is a crucial component, enabling the model to selectively focus on relevant parts of the MD signatures by calculating similarities between feature windows.

Datasets

The primary dataset used for training and evaluation is IDRad. This dataset was previously introduced and is based on FMCW radar recordings of human walking. The paper notes that for fair comparison, they used the same preprocessing as the original IDRad paper and input signatures of 205 Doppler channels and 45 time stamps. They also mention that their experiments were conducted on a validation split of IDRad.

Key Insights and Ideas

  • Gait as a Unique Identifier: The paper strongly emphasizes that unique gait characteristics, arising from the distinct movements of body parts (fast-moving limbs vs. slow-moving torso), are highly discriminative for human identification.
  • Dual-Scale Attention is Key: The success of DSDA hinges on its ability to process MD signatures at different scales. Attending separately to fast and slow components allows for a more comprehensive understanding of gait.
  • Fast-Moving Features are Highly Informative: Ablation studies show that Temporal Window Attention (TWA), which focuses on fast-moving features, significantly boosts performance, suggesting that limb movements contain crucial identifying information.
  • Interpretability through Attention: The paper highlights the interpretability of their attention mechanism, showing how TWA weights highlight specific windows corresponding to human walking movements.
  • Robustness to Dataset Scarcity: DSDA demonstrates good robustness even with reduced training data, suggesting it learns efficiently and is less sensitive to limited data compared to baseline methods.

Key Results

  • State-of-the-Art Performance: DSDA achieved a test error rate of 8.65% on the IDRad dataset, significantly outperforming previous methods.
  • Comparison to Baselines:
    • The original baseline [10] achieved a test error rate of 20.37% (reproduced with 150 time stamps).
  • An LSTM-based model achieved 18.42% test error, demonstrating the importance of sequential modeling.
  • DSDA's performance was substantially better than these, with a 10.87% error rate on the validation set.
  • Ablation Studies: Removing TWA or HWA individually led to significant performance degradation (error rates of 24.32% and 23.94%, respectively), underscoring the importance of both components.
  • Qualitative Analysis: Confusion matrices showed DSDA's improved performance across all targets, particularly in distinguishing challenging cases (e.g., target 4). Visualizations of attention weights confirmed that TWA effectively highlights windows corresponding to human walking.
  • Efficiency: DSDA shows faster error rate saturation during training compared to the baseline.

Limitations and Future Work

  • Limitations:
    • Task Specificity: The current work primarily focuses on human identification. While extended experiments suggest potential for human detection/localization, this is not the primary focus.
  • Dataset Dependency: The study is validated on a single dataset (IDRad). To establish generality, validation on more diverse radar datasets is needed.
  • Future Work:
    • Human Detection/Localization: Extend the current framework to tasks like human detection and localization within radar sequences.
  • Diverse Windowing Methods: Explore more advanced windowing mechanisms beyond the current sliding window approach.
  • Model Generality: Validate DSDA on additional radar datasets to ensure its robustness and generalizability as an attention module for radar signal understanding.

Summary updated

Abstract

This paper considers a Deep Convolutional Neural Network (DCNN) with an attention mechanism referred to as Dual-Scale Doppler Attention (DSDA) for human identification given a micro-Doppler (MD) signature induced as input. The MD signature includes unique gait characteristics by different sized body parts moving, as arms and legs move rapidly, while the torso moves slowly. Each person is identified based on his/her unique gait characteristic in the MD signature. DSDA provides attention at different time-frequency resolutions to cater to different MD components composed of both fast-varying and steady. Through this, DSDA can capture the unique gait characteristic of each person used for human identification. We demonstrate the validity of DSDA on a recently published benchmark dataset, IDRad. The empirical results show that the proposed DSDA outperforms previous methods, using a qualitative analysis interpretability on MD signatures.
BibTeX
@inproceedings{yoon2022dual,
  author = {Sunjae Yoon and Dahyun Kim and Ji Woo Hong and Junyeong Kim and Chang D. Yoo},
  title = {{Dual-Scale Doppler Attention for Human Identification}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=gpM1xQctL2u},
}
Figure