Vehicle Detection in Aerial Images Based on Lightweight Deep Convolutional Network and Generative Adversarial Network

Jiaquan Shen, Ningzhong Liu, Han Sun, Huiyu ZhouIEEE Access 2019 · OpenReview

Abstract

Vehicle detection in aerial imagery is a critical yet challenging task due to high computational costs and the scarcity of annotated data. This paper proposes a two-pronged solution: LD-CNNs, a lightweight deep convolutional neural network designed for efficient small object detection, and MC-GAN, a multi-condition constrained generative adversarial network that creates high-quality synthetic training data. The proposed model achieves state-of-the-art performance on the Munich public dataset with a mean Average Precision (mAP) of 86.9% and a detection time of 1.64s, significantly outperforming traditional deep learning models in both speed and efficiency.

The Challenge of Aerial Detection

Vehicle detection from above is often referred to as a "Goldilocks" problem. Objects are typically too small for standard detectors to identify clearly, backgrounds are cluttered with complex urban textures and shadows, and high-quality labeled datasets are expensive to produce.

While heavy-duty models like Faster R-CNN are accurate, they are computationally intensive. Furthermore, their aggressive downsampling (pooling) layers often discard the fine-grained details required to identify a vehicle that may only be a few dozen pixels wide. This research addresses these bottlenecks by optimizing the network architecture for small objects and automating data generation.

Proposed Methodology

The authors introduce a framework that integrates synthetic data generation with an optimized detection pipeline.

The overall framework of the proposed system, integrating MC-GAN for data generation and LD-CNNs for detection.

1. MC-GAN: Multi-Condition Constrained GAN

To overcome the shortage of labeled aerial images, the authors developed MC-GAN. Unlike standard GANs, it uses a double discriminator strategy. One discriminator ($D_c$) focuses on the realism of the generated vehicle, while a second background discriminator ($D_b$) ensures the vehicle blends seamlessly into the road texture and lighting of the aerial scene without creating edge artifacts.

2. LD-CNNs: Lightweight Deep CNNs

The core detection backbone, LD-CNNs, is custom-built for small objects. Standard networks like VGG16 or ResNet often double the number of channels at each stage, which can lead to "information jumps" that lose small-scale features.

Detailed architecture of the LD-CNNs backbone showing the Stem and Stack blocks designed for gradual feature refinement.

As shown in the architecture diagram, LD-CNNs utilize:

  • Stem Blocks: These use two-channel parallel propagation (1x1 and 3x3 convolutions) to capture multi-scale features immediately.
  • Stack Blocks: Instead of aggressive doubling, these blocks use gradual channel stacking, increasing channels by only 32 at a time. This allows the network to refine features more delicately, preserving the tiny signals of small vehicles.

Key Results and Insights

The researchers evaluated their approach on the Munich Public Dataset and a newly collected dataset from Nanjing, China.

  • Efficiency and Speed: The LD-CNN backbone is remarkably lean, featuring only 1.03 Million parameters (compared to 138M for VGG16). This efficiency allows for a detection time of 1.64s per image on standard hardware.
  • Accuracy: The model achieved 86.9% mAP on the Munich dataset, outperforming established benchmarks like SSD and YOLOv3.
  • The GAN Boost: Ablation studies revealed that including MC-GAN generated samples improved detection accuracy by 3.3%, proving that synthetic data can effectively bridge the gap in limited real-world datasets.

Conclusion

By combining a lightweight architecture optimized for small objects with a sophisticated GAN for data augmentation, this research provides a robust solution for near-real-time aerial vehicle detection. While some synthetic distortion remains a challenge, the model's ability to achieve state-of-the-art accuracy with minimal parameters marks a significant step forward for edge-based aerial surveillance applications.

Summary updated

Abstract

Vehicle detection in aerial images is a challenging task and plays an important role in a wide range of applications. Traditional detection algorithms are based on sliding-window searching and shallow-learning-based features, which limits the ability to represent features and generates a lot of computational costs. Recently, with the successful application of convolutional neural network in computer vision, many state-of-the-art detectors have been developed based on deep CNNs. However, these CNN-based models still face some difficulties and challenges in vehicle detection in aerial images. Firstly, the CNN-based detection model requires extensive calculations during training and detection, and the accuracy of detection for small objects is not high. In addition, deep learning models often require a large amount of sample data to train a robust detection model, while the annotated data of aerial vehicles is limited. In this study, we propose a lightweight deep convolutional neural network detection model named LD-CNNs. The detection algorithm not only greatly reduces the computational costs of the model, but also significantly improves the accuracy of the detection. What's more, in order to cope with the problem of insufficient training samples, we develop a multi-condition constrained generative adversarial network named MC-GAN, which can effectively generate samples. The detection performance of the proposed model has been evaluated on the Munich public dataset and the collected dataset respectively. The results show that on the Munich dataset, the proposed method achieves 86.9% on mAP (mean average precision), F1-score is 0.875, and the detection time is 1.64s on Nvidia Titan XP. At present, these detection indicators have reached state-of-the-art level in vehicle detection of aerial images.
BibTeX
@inproceedings{shen2019vehicle,
  author = {Jiaquan Shen and Ningzhong Liu and Han Sun and Huiyu Zhou},
  title = {{Vehicle Detection in Aerial Images Based on Lightweight Deep Convolutional Network and Generative Adversarial Network}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=Bu89gOkMLY},
}
Figure