Real-Time Concealed Weapon Detection on 3D Radar Images for Walk-Through Screening System

Nagma S. Khan, Kazumine Ogura, Eric Cosatto, Masayuki AriyoshiWACV 2023

TL;DR

This paper introduces a novel 3D Convolutional Neural Network (CNN) framework for real-time concealed weapon detection (CWD) using 3D radar images. By reformulating a 2D U-Net architecture into 3D and training it to predict peak-shaped probability maps, the system achieves both the accuracy and speed required for walk-through security screening.

What Problem Does This Paper Address?

The increasing need for security in public spaces has driven the development of automated concealed weapon detection (CWD) systems. Traditional body scanners, often used in airports, require individuals to remain stationary, leading to low throughput and making them unsuitable for crowded environments like train stations or malls. Walk-through screening systems offer a solution by allowing people to move through a scanning gate. However, these systems face two major challenges:

  1. Real-time Processing: To avoid missing threats, the system must process scans at a high frame rate (e.g., 20fps).
  2. Accuracy with Occlusion and Orientation: As individuals walk, their body parts can occlude weapons, and weapons can be presented in various 3D orientations. Conventional 2D projection methods, often used in existing CWD approaches, lose crucial shape information in these scenarios, especially with the lateral scanning used in walk-through systems.

Existing 3D object detection methods are often too computationally expensive or not designed for the ordered voxel data produced by radar imaging. This paper aims to bridge this gap by developing a system that is both accurate and fast enough for practical walk-through CWD.

Methods

The core of the proposed solution is a 3D U-Net based segmentation network adapted for a detection task. Here's a breakdown of the key methodological innovations:

  • 3D Radar Image Input: Instead of relying on 2D projections, the system directly processes the 3D radar images, preserving spatial relationships and shape information.
  • Peak-Shaped Probability Map Prediction: The network is trained to output a 3D probability map where a "peak" signifies the presence and location of a weapon. This is a departure from traditional segmentation which outputs voxel-wise masks. A Gaussian function is used to model these peaks, with its parameters adjusted to fit the weapon's bounding box.
  • Efficient Peak Detection: Weapon detection then becomes a simple peak detection operation on the predicted probability map. This is computationally efficient and allows for fast inference.
  • 3D U-Net Architecture: The standard 2D U-Net architecture is extended to 3D by replacing all 2D operations (convolution, pooling, etc.) with their 3D counterparts. To meet real-time constraints, the network's complexity is reduced by halving the feature map counts, resulting in a significantly smaller model.
  • Loss Function: A weighted loss function is employed to handle the class imbalance inherent in CWD, where weapon voxels constitute a small fraction of the total volume.

Datasets

The authors collected a new radar image dataset specifically for their walk-through weapon screening system. This dataset consists of 3D radar images from subjects walking through the system, either carrying concealed weapons (with-weapon scenarios) or everyday items (no-weapon scenarios). The dataset includes:

  • With-weapon scenarios: Weapons (pistol and revolver) concealed in realistic positions like bags or at the waist.
  • No-weapon scenarios: Subjects carrying items like cell phones, laptops, and water bottles.

The data distribution for training, testing, and validation is provided in Table 2, with 9800 with-weapon images and 10000 no-weapon images for training, and 2400 and 2500 respectively for testing.

Key Insights and Ideas

  • 3D is Crucial for Walk-Through CWD: The paper strongly argues that 2D projections are insufficient for walk-through screening due to significant loss of weapon shape information caused by varying orientations and lateral scanning. Direct 3D processing is essential.
  • Peak Maps for Detection: Shifting from voxel-wise masks to peak-shaped probability maps is a clever way to reframe segmentation as a detection problem. This allows for efficient localization via simple peak detection, which is much faster than processing dense masks.
  • Optimized U-Net for Real-Time: Adapting the U-Net architecture to 3D and then aggressively reducing its complexity (feature map counts) is key to achieving the required 20fps frame rate without sacrificing too much accuracy.
  • "Difficult" Cases Highlight 3D Advantage: The distinction between "easy" and "difficult" cases (based on how well weapon shape is preserved in 2D projections) clearly demonstrates the superiority of the 3D approach, especially when shape information is compromised.

Key Results

  • Real-time Performance: The proposed 3D approach achieves a run-time of 20fps, meeting the system's requirement.
  • High Accuracy: The system achieves an AUC_eff of 88.8%, outperforming other candidate architectures, including 2D projection-based methods and even a 3D RPN approach in terms of the accuracy-runtime trade-off.
  • Superiority over 2D: In comparisons, the proposed 3D method consistently outperforms 2D projection-based approaches (e.g., multi-view, z_int, x_int), especially in "difficult" scenarios where weapon shape is less preserved in 2D.
  • Peak Map Advantage: The peak-shaped probability map approach significantly outperforms the traditional voxel-wise binary mask approach for detection.
  • Optimal Parameter Selection: The study identifies an optimal proportionality factor (k=0.25) for the Gaussian function used in probability map generation, balancing coverage of the weapon shape with avoiding contextual noise.

Limitations and Future Work

  • Single Weapon Focus: The current paper primarily focuses on detecting a single weapon per image. The authors note that their approach is capable of localizing multiple weapons via multi-peak detection, but the detailed analysis and experiments are for single weapon scenarios. Extending to robust multi-weapon detection would be a natural next step.
  • Dataset Specificity: While the dataset is realistic for their prototype, its size and diversity might be a limitation for generalization to all possible weapon types and concealment methods.
  • Computational Cost of 3D: Although optimized, 3D processing is inherently more computationally intensive than 2D. Further research into more efficient 3D architectures or hardware acceleration could be beneficial.
  • Generalization to Other Objects: The current system is trained for weapon detection. Future work could explore adapting this framework for detecting other types of contraband or threats.

Summary updated

BibTeX
@inproceedings{khan2023real,
  author = {Nagma S. Khan and Kazumine Ogura and Eric Cosatto and Masayuki Ariyoshi},
  title = {{Real-Time Concealed Weapon Detection on 3D Radar Images for Walk-Through Screening System}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure