Weakly Supervised Rotation-Invariant Aerial Object Detection Network

Xiaoxu Feng, Xiwen Yao, Gong Cheng, Junwei HanCVPR 2022 · OpenReview

Abstract

Object rotation is among longstanding, yet still unexplored, hard issues encountered in the task of weakly supervised object detection (WSOD) from aerial images. Existing predominant WSOD approaches built on regular CNNs which are not inherently designed to tackle object rotations without corresponding constraints, thereby leading to rotation-sensitive object detector. Meanwhile, current solutions have been prone to fall into the issue with unstable detectors, as they ignore lower-scored instances and may regard them as backgrounds. To address these issues, in this paper, we construct a novel end-to-end weakly supervised Rotation-Invariant aerial object detection Network (RINet). It is implemented with a flexible multi-branch online detector refinement, to be naturally more rotation-perceptive against oriented objects. Specifically, RINet first performs label propagating from the predicted instances to their rotated ones in a progressive refinement manner. Meanwhile, we propose to couple the predicted instance labels among different rotation-perceptive branches for generating rotation-consistent supervision and meanwhile pursuing all possible instances. With the rotation-consistent supervisions, RINet enforces and encourages consistent yet complementary feature learning for WSOD without additional annotations and hyper-parameters. On the challenging NWPU VHR-10.v2 and DIOR datasets, extensive experiments clearly demonstrate that we significantly boost existing WSOD methods to a new state-of-the-art performance. The code is available at: https://github.com/XiaoxFeng/RINet.

Introduction

Weakly Supervised Object Detection (WSOD) aims to train object detectors using only image-level category labels, significantly reducing the annotation burden. However, applying WSOD to aerial imagery introduces unique challenges. Unlike natural images, aerial objects like aircraft, ships, and storage tanks appear in arbitrary orientations. Standard CNN-based WSOD models often lack rotation invariance, leading to inconsistent detections when the image is rotated. Furthermore, these models frequently suffer from the "discriminative part" problem, where they only localize the most salient part of an object (e.g., a plane's cockpit) or miss multiple instances in a single scene.

Visualizing the challenges of rotation sensitivity and missing instances in aerial WSOD, and how RINet addresses them.

Methodology: The RINet Framework

To overcome these hurdles, the proposed Rotation-Invariant Network (RINet) introduces two core innovations built upon the Online Instance Classifier Refinement (OICR) framework.

1. Rotation-Invariant Learning

RINet employs a multi-branch architecture to enforce rotation consistency. During training, the network processes both the original image and its rotated version. It consists of three primary branches:

  • Detection Branch: Handles the original image features.
  • Rotated Detection Branch: Processes features from the rotated image.
  • Rotation-Invariant Branch: Couples predictions from both branches to generate consistent pseudo-labels.

By requiring the model to produce identical labels for an object regardless of its orientation, RINet learns features that are inherently more robust to rotation.

The architecture of RINet, featuring multi-branch rotation-invariant learning and graph-based instance mining.

2. Multiple Instance Mining (MIM)

To address the issue of missing instances, RINet utilizes a graph-based label propagation strategy. Object proposals are clustered based on their scores, and a graph is constructed to connect proposals across original and rotated views. This allows the model to propagate high-confidence labels to lower-scoring but geometrically consistent proposals, effectively "mining" all potential instances in a scene rather than just the most salient one.

Experimental Results

The performance of RINet was evaluated on two benchmark aerial datasets: NWPU VHR-10.v2 and DIOR.

  • NWPU VHR-10.v2: RINet achieved a mean Average Precision (mAP) of 70.4%, a substantial improvement over the 34.5% mAP of the OICR baseline.
  • DIOR: On this larger, more complex dataset, RINet reached 28.3% mAP and 52.8% CorLoc, setting a new state-of-the-art for weakly supervised methods.

Qualitative analysis confirms that RINet produces more complete and consistent bounding boxes across various orientations compared to previous methods, which often failed under rotation or localized only partial objects.

Conclusion and Future Work

RINet successfully addresses rotation sensitivity and instance missing in aerial WSOD through self-supervised geometric constraints and graph-based mining. While it sets a new performance benchmark, challenges remain in detecting very small objects and resolving contextual ambiguities (e.g., confusing bridges with the rivers they span). Future research may explore causal intervention techniques to better decouple objects from their typical backgrounds.

Summary updated

Abstract

Object rotation is among longstanding, yet still unexplored, hard issues encountered in the task of weakly supervised object detection (WSOD) from aerial images. Existing predominant WSOD approaches built on regular CNNs which are not inherently designed to tackle object rotations without corresponding constraints, thereby leading to rotation-sensitive object detector. Meanwhile, current solutions have been prone to fall into the issue with unsTable detectors, as they ignore lower-scored instances and may regard them as backgrounds. To address these issues, in this paper, we construct a novel end-to-end weakly supervised Rotation-Invariant aerial object detection Network (RINet). It is implemented with a flexible multi-branch online detector refinement, to be naturally more rotation-perceptive against oriented objects. Specifically, RINet first performs label propagating from the predicted instances to their rotated ones in a progressive refinement manner. Meanwhile, we propose to couple the predicted in-stance labels among different rotation-perceptive branches for generating rotation-consistent supervision and mean-while pursuing all possible instances. With the rotation-consistent supervisions, RINet enforces and encourages consistent yet complementary feature learning for WSOD without additional annotations and hyper-parameters. On the challenging NWPU VHR-10.v2 and DIOR datasets, extensive experiments clearly demonstrate that we significantly boost existing WSOD methods to a new state-of-the-art performance. The code will be available at: https://github.com/XiaoxFeng/RINet.
BibTeX
@inproceedings{feng2022weakly,
  author = {Xiaoxu Feng and Xiwen Yao and Gong Cheng and Junwei Han},
  title = {{Weakly Supervised Rotation-Invariant Aerial Object Detection Network}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  url = {https://openreview.net/forum?id=b5knYLX6HsI},
}
Figure