HS-FPN: High Frequency and Spatial Perception FPN for Tiny Object Detection

Zican Shi, Jing Hu, Jie Ren, Hengkang Ye, Xuyang Yuan, Yan Ouyang, Jia He, Bo Ji, Junyu GuoAAAI 2025 · DOI

TL;DR

This paper introduces HS-FPN, a novel Feature Pyramid Network designed to improve the detection of tiny objects. By incorporating a High Frequency Perception module to enhance subtle features and a Spatial Dependency Perception module to capture contextual relationships, HS-FPN significantly boosts performance on challenging tiny object detection tasks.

What Problem Does This Paper Address?

Detecting "tiny objects" – defined as objects smaller than 16x16 pixels – is a critical but notoriously difficult task in computer vision. These objects are prevalent in applications like traffic sign recognition, autonomous driving, and aerial surveillance. Existing object detection frameworks, even those enhanced by Feature Pyramid Networks (FPNs), struggle with tiny objects for several reasons:

  • Limited Usable Features: Tiny objects occupy very few pixels, and the downsampling inherent in deep learning models further reduces their feature representation, making them weak and easily lost.
  • Lack of Network Attention: The weak features of tiny objects are easily overwhelmed by background noise and interference. Standard FPNs treat all features equally, failing to provide the specialized attention tiny objects require.
  • Poor Spatial Perception: FPNs merge features from different levels in a pixel-wise manner, which can lead to misalignment between features from different scales, especially for small objects. This limits their ability to understand the spatial context around tiny targets.

Methods

The proposed HS-FPN builds upon the standard FPN architecture by introducing two novel modules:

  1. High Frequency Perception Module (HFP): This module aims to enhance the discriminative features of tiny objects. It works by:
  2. High-Pass Filtering: Applying a high-pass filter to the input feature maps to extract high-frequency components, which are characteristic of object edges and details, while suppressing low-frequency background noise.
  3. Channel Path (CP): Dynamically assigning weights to different feature channels based on their high-frequency responses, highlighting channels that are more relevant to tiny objects.
  4. Spatial Path (SP): Generating a spatial attention mask that guides the network to focus on regions containing tiny object features.
  5. Spatial Dependency Perception Module (SDP): This module addresses the lack of spatial context in FPNs. It leverages an attention mechanism inspired by Vision Transformers (ViT) but adapted for feature maps:
  6. Cross-Attention: It computes attention between pixels from the lower-level feature map ($C_i$) and the upsampled higher-level feature map ($P_{i+1}$). This allows pixels from $C_i$ to interact with a wider range of pixels in $P_{i+1}$, capturing long-range spatial dependencies.
  7. Feature Block Processing: Unlike ViT's block-wise attention, SDP processes features in smaller blocks and computes cross-attention within these blocks, effectively enriching tiny object features with contextual information from adjacent areas.

These modules are integrated into the lateral connections of the FPN, allowing them to refine features before they are merged.

Datasets

The primary dataset used for training and evaluation is AI-TOD (Wang et al. 2021), a challenging dataset specifically designed for tiny object detection in aerial images. The authors also created a subset called DOT Amini10 from the DOTA dataset, which contains a significant proportion of small objects.

Key Insights and Ideas

  • Frequency Domain for Tiny Objects: The paper highlights that tiny objects, being detailed and edge-like, reside in the high-frequency components of an image. By filtering out low frequencies, these subtle features can be amplified, leading to better detection.
  • Synergy of HFP and SDP: The combination of HFP and SDP proves more effective than either module alone. HFP enhances the features themselves, while SDP helps contextualize these enhanced features, leading to a more robust representation.
  • Attention Adaptation: Adapting the attention mechanism from ViT to operate on feature maps and within feature blocks (rather than across blocks) is crucial for capturing fine-grained spatial dependencies relevant to tiny objects.
  • Modular Design: HS-FPN is designed to be modular, allowing it to replace standard FPNs in various existing object detection frameworks with minimal architectural changes.

Key Results

  • Significant Performance Gains: HS-FPN consistently outperforms baseline FPN-based detectors. For instance, using Cascade R-CNN with ResNet50 as a backbone, HS-FPN achieves an AP of 23.6, a notable improvement over FPN's 20.2 on the AI-TOD dataset.
  • Component Effectiveness: Ablation studies confirm the contributions of both HFP and SDP. HFP alone improves AP by 2.2 points, while SDP adds 1.1 points. Their combination yields a substantial 3.4-point increase in AP compared to FPN.
  • Broad Applicability: HS-FPN shows improvements across various backbone networks (ResNet101, MobileNetV2) and detection frameworks (Faster R-CNN, Cascade R-CNN, DetectoRS), demonstrating its versatility.
  • Minimal Computational Overhead: While HS-FPN slightly increases computational cost, the gains in detection performance are substantial, making it a worthwhile trade-off.

Limitations and Future Work

  • Parameter Tuning: The effectiveness of the HFP module is sensitive to the hyperparameter $\alpha$ that controls the high-pass filter's frequency band. The paper finds $\alpha=0.25$ to be optimal for the tested datasets.
  • Noise Amplification: While HFP enhances tiny object features, it can also amplify high-frequency noise. The SDP module helps to suppress some of this amplified noise by incorporating contextual information.
  • Future Work: The paper mentions an "extended version" which likely contains further details, but specific future work directions are not explicitly detailed in the provided text. However, the success of HS-FPN suggests its application to other challenging detection scenarios beyond tiny objects.

Summary updated

BibTeX
@inproceedings{shi2025hs,
  author = {Zican Shi and Jing Hu and Jie Ren and Hengkang Ye and Xuyang Yuan and Yan Ouyang and Jia He and Bo Ji and Junyu Guo},
  title = {{HS-FPN: High Frequency and Spatial Perception FPN for Tiny Object Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i7.32740},
}
Figure