Malicious UAV Detection Using Integrated Audio and Visual Features for Public Safety Applications

Sonain Jamil, Fawad, MuhibUr Rahman, Amin Ullah, Salman Badnava, Masoud Forsat, Seyed Sajad Mirjavadi2020 · DOI

TL;DR

This paper introduces a novel hybrid approach for detecting and localizing malicious drones by combining audio and visual features. By integrating handcrafted audio descriptors with deep visual features, the system achieves high accuracy even in challenging conditions like noise and low resolution, making it a promising tool for public safety applications.

What Problem Does This Paper Address?

Unmanned Aerial Vehicles (UAVs), or drones, have become ubiquitous for surveillance, security, and remote monitoring. While beneficial, their increasing prevalence also poses significant security threats, particularly concerning public privacy and the potential for malicious use. Detecting and localizing these "malicious" drones (those carrying illicit payloads, collecting unauthorized data, or entering restricted zones) in a timely and accurate manner is a critical, yet challenging, open research problem. Existing methods often struggle with limitations such as extreme weather conditions, the need for massive datasets, or poor performance in noisy environments. This paper aims to address this gap by developing a robust and efficient detection system.

Methods

The proposed methodology employs a two-pronged approach: audio feature extraction and visual feature extraction, followed by classification.

  • Audio Feature Extraction: Mel Frequency Cepstrum Coefficients (MFCC) are used to extract features from audio samples. This process involves pre-emphasis, windowing, Fast Fourier Transform, application of Mel filter banks, logarithmic energy calculation, and finally, Discrete Cosine Transform (DCT) to obtain the MFCCs.
  • Visual Feature Extraction: The deep learning model AlexNet is utilized for image feature extraction. AlexNet, with its 25 layers (including convolutional, max-pooling, and fully connected layers), processes input images to generate feature vectors.
  • Classification: A Support Vector Machine (SVM) is employed as the classifier. The paper explores linear, Gaussian, and polynomial kernels for SVM to find the optimal configuration for distinguishing between malicious drones and other objects. The SVM's hyperplane is trained to minimize classification error.
  • Localization: For localization, a scale pyramid approach is used with the input image. Patches are extracted from different scales with overlap, features are extracted using AlexNet, and if identified as a drone, its coordinates are scaled back to the original image space to annotate a bounding box.

Datasets

The research utilizes two primary datasets:

  • Image Dataset: A dataset of 506 images was used, with 70% for training and 30% for testing. This dataset includes five classes: birds, airplanes, kites, balloons, and drones. The images are designed to represent various flight scenarios (low/high altitude, different weather conditions, visibility levels) and variations in resolution, scale, orientation, and illumination, with some drone images featuring environmental occlusions.
  • Audio Dataset: A dataset of 217 audio samples was used, with 70% for training and 30% for testing. This dataset comprises audio samples of drones, airplanes, birds, and thunderstorms, with varying lengths. Spectrograms are provided to visualize the frequency characteristics of these sounds.

The paper also mentions a combined dataset of images and audio samples for multiclass classification, containing four classes: Drones, Thunder, Birds, and Planes.

Key Insights and Ideas

  • Hybrid Approach Superiority: The core insight is that combining audio and visual features significantly enhances detection accuracy and robustness compared to using either modality alone. This is particularly effective in overcoming the limitations of individual methods (e.g., visual methods failing in adverse weather, audio methods being susceptible to noise).
  • Deep Learning for Visuals, MFCC for Audio: The paper demonstrates the effectiveness of AlexNet for visual feature extraction and MFCC for audio feature extraction, showcasing their suitability for this specific task.
  • SVM's Versatility: The study highlights SVM's capability to effectively classify complex feature sets, even those with high dimensionality, and its performance across different kernel types.
  • Cost-Effectiveness with Small Datasets: A key advantage of the proposed method is its ability to achieve high accuracy with a relatively small dataset, making it more practical and cost-effective for deployment.

Key Results

  • Visual Detection: AlexNet, when used with a polynomial kernel of SVM, achieved an accuracy of 97.4% for visual drone detection.
  • Audio Detection: MFCC, when used with a Gaussian kernel of SVM, achieved a high precision rate of 98.3% for detecting drones based on sound, even in noisy environments.
  • Combined Approach: The integrated audio-visual approach using combined MFCC and AlexNet features yielded the highest accuracy of 98.5%.
  • Localization: The proposed localization method, using AlexNet features, was able to annotate detected drones with bounding boxes.
  • Computational Efficiency: The model demonstrates reasonable computational times, with image feature extraction taking 1.16s and audio feature extraction taking 0.3s per sample. Training the combined model took 30 minutes, and classification of objects took within 2 seconds.

Limitations and Future Work

  • Limitations: The paper acknowledges that challenges such as low resolution, occlusion, and noisy audio were addressed, but these remain inherent difficulties. While the proposed method performs well, it's noted that previous approaches did not consider these challenges.
  • Future Work: The authors plan to include Recurrent Convolutional Neural Network (RCNN) techniques and wireless communication aspects in future iterations of the proposed model.

Summary updated

BibTeX
@misc{jamil2020malicious,
  author = {Sonain Jamil and Fawad and MuhibUr Rahman and Amin Ullah and Salman Badnava and Masoud Forsat and Seyed Sajad Mirjavadi},
  title = {{Malicious UAV Detection Using Integrated Audio and Visual Features for Public Safety Applications}},
  year = {2020},
  doi = {10.3390/s20143923},
}
Figure