Renormalized Connection for Scale-preferred Object Detection in Satellite Imagery

Fan Zhang, Lingling Li, Licheng Jiao, Xu Liu, Fang Liu, Shuyuan Yang, Biao Hou

Abstract

Detecting tiny objects in satellite imagery, such as vehicles or small vessels, remains a significant challenge for modern deep learning architectures. Standard Feature Pyramid Networks (FPNs) often suffer from "interfering activations"—background noise in high-level branches that mimics large objects and overwhelms the signal of small targets. This paper introduces Renormalized Connections (RC), a lightweight, physics-inspired plug-in designed to harmonize multi-scale features. By applying principles from renormalization group theory, RC enables features to work synergistically rather than independently. Experimental results across five diverse datasets demonstrate that RC significantly boosts detection accuracy for tiny objects with virtually zero additional computational cost.

The Challenge of Scale-Preferred Detection

In remote sensing, objects of interest are frequently just a few pixels in size. Traditional object detectors employ a "divide-and-conquer" strategy via FPNs, where different network layers are assigned to detect objects of different scales. However, for "scale-preferred" tasks—where the objective is exclusively tiny objects—this strategy falters.

The authors observed that high-level FPN branches, intended for large objects, generate massive amounts of background noise (e.g., from roads or buildings) when no large objects are present. This noise interferes with the training of small-object branches, a problem that standard loss functions like Focal Loss cannot adequately resolve.

Methodology: Renormalized Connections

To address this, the researchers proposed a two-step solution inspired by Renormalization Group Theory, which studies physical systems at varying scales.

1. Knowledge Discovery Network (KDN)

The team first developed a single-branch "probe" network to analyze the relationships between features at different backbone stages. They discovered that while these stages are structurally independent, they can be combined to act synergistically to suppress noise and enhance target signals.

2. Renormalized Connection (RC) Architecture

Based on KDN findings, the authors abstracted a linear connection logic termed n21C. The most effective variant, 421C (or E421C), uses a specific 4:2:1 ratio to combine features from different levels. Unlike simple feature addition, RC constructs "independent bases" by subtracting correlated parts of different layers. This ensures the information reaching the detection head is "renormalized"—cleared of redundant background noise and focused on the target scale.

The Renormalized Connection (RC) architecture acts as a bridge between the feature extractor and detection levels, enabling synergistic feature fusion.

Experimental Results and Insights

The effectiveness of RC was validated across five datasets, including the new IPIU Dataset (focused on tiny moving vehicles) and standard benchmarks like RSOD and NWPU VHR-10.

Noise Suppression

Visual analysis using heatmaps confirms that the standard PAFPN architecture produces significant interfering activations in its high-level branches (P4 and P5). When the 421C connection is integrated, these false activations are drastically reduced, allowing the model to focus exclusively on the actual targets.

Heatmap comparison between standard PAFPN and PAFPN with 421C. Red boxes highlight interfering background noise that is successfully suppressed by the Renormalized Connection.

Performance Gains

  • Accuracy: On the challenging IPIU task, adding 421C to Faster R-CNN improved mAP from 13.4% to 26.3%. For YOLOv8, mAP rose from 55.1% to 55.5%.
  • Efficiency: The linear E421C version adds almost no parameters. YOLOv8 with RC maintained a real-time inference speed of 16.89 ms on a GTX 1080 GPU.
  • Versatility: The method proved robust across 17 different architectures, including FCOS, RetinaNet, and Swin Transformers.

Conclusion

Renormalized Connections offer a simple yet powerful "Occam’s Razor" solution for satellite object detection. By reorienting gradient flow and filtering redundant noise through physics-inspired linear combinations, RC allows standard detectors to excel at tiny object detection without the need for heavy architectural redesigns. Future work will explore applying these connections to even more complex multi-scale remote sensing missions.

Summary updated

Abstract

Satellite imagery, due to its long-range imaging, brings with it a variety of scale-preferred tasks, such as the detection of tiny/small objects, making the precise localization and detection of small objects of interest a challenging task. In this article, we design a Knowledge Discovery Network (KDN) to implement the renormalization group theory in terms of efficient feature extraction. Renormalized connection (RC) on the KDN enables ``synergistic focusing'' of multi-scale features. Based on our observations of KDN, we abstract a class of RCs with different connection strengths, called n21C, and generalize it to FPN-based multi-branch detectors. In a series of FPN experiments on the scale-preferred tasks, we found that the ``divide-and-conquer'' idea of FPN severely hampers the detector's learning in the right direction due to the large number of large-scale negative samples and interference from background noise. Moreover, these negative samples cannot be eliminated by the focal loss function. The RCs extends the multi-level feature's ``divide-and-conquer'' mechanism of the FPN-based detectors to a wide range of scale-preferred tasks, and enables synergistic effects of multi-level features on the specific learning goal. In addition, interference activations in two aspects are greatly reduced and the detector learns in a more correct direction. Extensive experiments of 17 well-designed detection architectures embedded with n21s on five different levels of scale-preferred tasks validate the effectiveness and efficiency of the RCs. Especially the simplest linear form of RC, E421C performs well in all tasks and it satisfies the scaling property of RGT. We hope that our approach will transfer a large number of well-designed detectors from the computer vision community to the remote sensing community.
BibTeX
@article{zhang2024renormalized,
  author = {Fan Zhang and Lingling Li and Licheng Jiao and Xu Liu and Fang Liu and Shuyuan Yang and Biao Hou},
  title = {{Renormalized Connection for Scale-preferred Object Detection in Satellite Imagery}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2024},
  doi = {10.1109/tgrs.2024.3440881},
}
Figure