Abstract
This paper proposes a novel RHF-Net (Recursive Hybrid Fusion pyramid network) to solve the problem of small object detection on real-time embedded devices. Though the object detection accuracy rate is improved by a large margin with state-of-the-art models, e.g., SSD, YOLO, RetinaNet, and RefineDet, they are still problematic for small object detection and inefficient on embedded systems. One novelty of the RHF-Net is a bidirectional fusion module that allows to fuse feature maps with both the top-down and bottom-up directions to generate flexible FPs for small object detection. This module can be easily integrated to any feature pyramid based object detection model. Another novelty of this net is a recursive concatenation and reshaping module which can recursively concatenate not only high-level semantic features from deep layers but also reshape spatially richer features from shallower layers to prevent small objects from disappearing. RHF-Net adopts computationally low-cost and feature preserving operations in the fusion, thus it is efficient and accurate even on embedded devices. The superiority of RHF-Net is investigated on the COCO benchmark and UAVDT dataset in terms of mAP and FPS.
Introduction
Detecting small objects—typically those smaller than 32x32 pixels—is a notorious challenge for deep learning models. Standard convolutional neural networks (CNNs) downsample images as they go deeper to extract high-level semantic meaning. However, for tiny objects, this downsampling often reduces them to a single pixel or eliminates them entirely before they reach the prediction layer.
While Feature Pyramid Networks (FPNs) attempt to address this by fusing deep semantic features with shallower spatial features, most are one-directional (top-down) and computationally expensive. This makes them unsuitable for real-time applications on power-constrained embedded devices, such as drones or smart traffic sensors, where low latency is critical.
Methodology
The RHF-Net architecture introduces two primary innovations to create a more efficient and "small-object-aware" network: the RESC block and the Hybrid Bottom-up Fusion module.
RESC (REShaped and Concatenated) Block
Standard models often use pooling or addition to combine layers. Pooling discards information, and addition requires computationally expensive 1x1 convolutions to match channel dimensions. The RESC block instead reshapes spatially rich features from shallow layers (reorganizing pixels into channels) and concatenates them. This preserves contextual information without the overhead of extra convolutions, maintaining the "feature richness" critical for discriminating tiny objects from their backgrounds.
Hybrid Bottom-up Fusion Module
Most FPNs only pass information from deep layers back to shallow ones (top-down). RHF-Net adds a bottom-up path, allowing the network to recursively bring spatially richer features from the start of the network back into the final prediction layers. This ensures that small objects that might have "disappeared" in the deep layers are re-introduced for the final detection.
Architecture Overview
The RHF-Net is flexible and can be integrated with various backbones, such as DarkNet-53 or the lightweight Pelee backbone. By utilizing concatenation—a feature-preserving operation—the network reduces the "feature-richness dithering" that occurs when merging disparate feature maps.
Experimental Evaluation
The model was evaluated on three benchmarks: MS COCO (general detection), UAVDT (drone footage with many tiny objects), and CarFlow (fisheye traffic cameras).
Performance on Embedded Hardware
On an NVIDIA Jetson TX2, RHF-Net (with a Pelee backbone) achieved 23 FPS, whereas standard state-of-the-art models like YOLOv3 or RefineDet were too slow for real-time use (often < 3 FPS). This demonstrates the efficiency of the proposed architecture for edge computing.
Small Object Detection Accuracy
On the UAVDT dataset, RHF-Net improved Average Precision (AP) significantly. For instance, adding the hybrid fusion to a DarkNet-53 backbone raised the AP from 59.39 to 62.18 with negligible impact on speed. On MS COCO, RHF-416 achieved 35.2 AP at 50 FPS (on a Titan X), outperforming YOLOv3-416 in both accuracy and speed.
Conclusion
RHF-Net addresses the limitations of traditional FPNs for small object detection on embedded devices. By employing a recursive reshape-and-concatenate strategy and a bidirectional fusion path, it maintains high feature richness and accuracy while remaining computationally efficient. Future work will explore further optimizations for edge computing and methods to improve robustness to image translations.
Summary updated