Clustered Object Detection in Aerial Images

Fan Yang, Heng Fan, Peng Chu, Erik Blasch, Haibin Ling

TL;DR

ClusDet (Clustered Detection) is a novel framework designed to solve the "needle in a haystack" problem of aerial object detection. By first predicting where objects are likely to cluster and then adaptively normalizing the scale of those regions, it significantly improves detection accuracy for tiny objects while drastically reducing redundant computation compared to traditional uniform tiling methods.

Introduction: The Challenge of Aerial Imagery

Detecting objects in aerial imagery, such as those captured by drones or satellites, presents unique challenges that standard object detectors struggle to overcome. The primary hurdles are:

  1. Extreme Scale Variation: Objects like pedestrians or vehicles often occupy only a few pixels. Detectors trained on standard datasets like COCO are not optimized for such minute scales.
  2. Sparsity and Non-uniformity: High-resolution aerial shots often contain vast areas of irrelevant background (e.g., rooftops, fields). Objects typically congregate in specific clusters, such as parking lots or sidewalks.

Traditional "brute force" methods involve dividing the image into a uniform grid of "chips." This approach is inefficient, as it processes many empty regions and can truncate objects that span grid boundaries.

Comparison of uniform grid-based partitioning versus the proposed cluster-wise partitioning, highlighting the increased density of object-rich chips.

Methodology: The ClusDet Framework

The authors propose ClusDet, an end-to-end pipeline that unifies clustering and detection. The architecture consists of three specialized sub-networks:

  • Cluster Proposal Sub-network (CPNet): Instead of immediate object detection, CPNet identifies "cluster regions." It functions as a modified Region Proposal Network (RPN) with an expanded receptive field to locate high-density areas.
  • Scale Estimation Sub-network (ScaleNet): After identifying a cluster, ScaleNet predicts the optimal scale for objects within that chip. Rather than simple resizing, it estimates a "scale offset" to bring objects into a size range where the detector is most effective.
  • Dedicated Detection Network (DetecNet): This stage performs fine-grained detection on scale-normalized chips. Because these chips are dense with objects, the detector can more effectively distinguish targets from the background.
The ClusDet pipeline, illustrating the flow from feature extraction and cluster proposal to scale estimation, detection, and final fusion.

To refine the output, ClusDet employs Iterative Cluster Merging (ICM) to manage overlapping proposals and a Local-Global Fusion strategy, combining high-resolution cluster detections with a low-resolution global pass.

Key Insights

  • Clustering as a Computational Filter: Grid-based partitioning results in over 73% "sparse" chips. ClusDet increases the density of "clustered" chips by 7x, focusing computational resources on object-rich areas.
  • Scale Normalization: Simply increasing resolution is insufficient. ScaleNet acts as an "adaptive zoom," standardizing object sizes to match the detector's learned anchors.
  • Contextual Advantage: By focusing on clusters, the model implicitly leverages environmental context (e.g., cars on roads), reducing false positives in unlikely locations like tree canopies.

Results and Evaluation

ClusDet was evaluated on major aerial benchmarks, including VisDrone, UAVDT, and DOTA.

  • Accuracy: On VisDrone, ClusDet achieved an Average Precision (AP) of 28.4%, outperforming standard Faster R-CNN (21.4%) and uniform tiling methods (24.4%).
  • Efficiency: The framework required approximately 40% fewer image forwards than grid-based methods while achieving higher accuracy.
  • Small Object Detection: The most significant gains were in the $AP_s$ metric, showing nearly a 2x improvement over baseline models.

Conclusion

ClusDet effectively addresses the inefficiencies of aerial object detection by prioritizing object clusters and adaptive scaling. While it excels at detecting grouped objects, future work may focus on better handling isolated objects and integrating temporal data from video streams to further enhance stability in dynamic environments.

Summary updated

Abstract

Detecting objects in aerial images is challenging for at least two reasons: (1) target objects like pedestrians are very small in pixels, making them hardly distinguished from surrounding background; and (2) targets are in general sparsely and non-uniformly distributed, making the detection very inefficient. In this paper, we address both issues inspired by observing that these targets are often clustered. In particular, we propose a Clustered Detection (ClusDet) network that unifies object clustering and detection in an end-to-end framework. The key components in ClusDet include a cluster proposal sub-network (CPNet), a scale estimation sub-network (ScaleNet), and a dedicated detection network (DetecNet). Given an input image, CPNet produces object cluster regions and ScaleNet estimates object scales for these regions. Then, each scale-normalized cluster region is fed into DetecNet for object detection. ClusDet has several advantages over previous solutions: (1) it greatly reduces the number of chips for final object detection and hence achieves high running time efficiency, (2) the cluster-based scale estimation is more accurate than previously used single-object based ones, hence effectively improves the detection for small objects, and (3) the final DetecNet is dedicated for clustered regions and implicitly models the prior context information so as to boost detection accuracy. The proposed method is tested on three popular aerial image datasets including VisDrone, UAVDT and DOTA. In all experiments, ClusDet achieves promising performance in comparison with state-of-the-art detectors. Code will be available in \url{https://github.com/fyangneil}.
BibTeX
@inproceedings{yang2019clustered,
  author = {Fan Yang and Heng Fan and Peng Chu and Erik Blasch and Haibin Ling},
  title = {{Clustered Object Detection in Aerial Images}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00840},
}
Figure