TL;DR
This paper tackles the challenge of detecting small objects on resource-constrained IoT devices by introducing an adaptive tiling method that enhances the signal-to-noise ratio for these tiny targets. They demonstrate significant improvements in detection accuracy and a reduction in object count errors, even outperforming larger models on embedded hardware.
What Problem Does This Paper Address?
The proliferation of Internet of Things (IoT) devices has opened up exciting possibilities for applications like remote monitoring and process automation. Many of these applications critically depend on accurately detecting objects, often very small ones, within a scene. However, current state-of-the-art object detection models, typically based on complex neural networks, are computationally intensive and require significant power, making them unsuitable for the low-power, memory-constrained microcontrollers common in IoT devices. Existing lightweight networks, while efficient, struggle with small object detection due to factors like limited pixel information and feature map downsampling that can cause small objects to disappear. This paper aims to bridge this gap by enabling robust small object detection on these embedded platforms.
Methods
The authors build upon the FOMO (Faster Objects More Objects) network, a lightweight architecture designed for microcontrollers that predicts object centers rather than bounding boxes. They introduce two key enhancements:
- Adaptive Tiling: To overcome the limited pixel information for small objects and the FOMO architecture's limitation of making only one prediction per grid cell, they propose splitting input images into smaller, overlapping tiles. This effectively increases the relative size of objects within each tile, improving the signal-to-noise ratio. The "adaptive" aspect comes from dynamically adjusting the number of tiles based on a target object size (quantified by Normalized Bounding Box Area or NBA), preventing unnecessary tiling and processing time.
- Soft F1 Loss: To address the issue of imbalanced datasets (where small objects are often underrepresented), they incorporate a soft F1 loss function. This loss metric is shown to better balance precision and recall, particularly when dealing with a heavy imbalance between objects and background.
Datasets
The primary dataset used for evaluation is the CARPK dataset. This dataset was collected by a drone flying at a moderate altitude over parking lots, containing approximately 1500 images with over 90,000 car annotations. It is specifically noted for its relevance to small object detection in aerial monitoring scenarios.
Key Insights and Ideas
- Tiling is Crucial for Small Objects: The paper strongly suggests that the primary hurdle for small object detection on embedded devices is the inherent lack of information available for these tiny targets. Tiling, by increasing the relative object size, directly addresses this.
- Adaptive Tiling Balances Performance and Efficiency: The adaptive nature of their tiling method is key. It allows for increased detection performance by using more tiles when needed but avoids excessive computational overhead by scaling the tiling based on object size estimations.
- Soft F1 Loss for Imbalanced Data: The soft F1 loss proves effective in mitigating the negative impacts of imbalanced datasets, leading to a better balance between precision and recall, which is vital for reliable object detection.
- Feasibility on Microcontrollers: The work demonstrates that even with these enhancements, the proposed approach can be quantized and run effectively on low-power microcontrollers like the Sony Spresense, achieving a good trade-off between detection performance, latency, and memory consumption.
Key Results
The proposed adaptive tiling method, combined with the soft F1 loss, yielded significant improvements on the CARPK dataset:
- Object Count Error Reduction: A 76% reduction in object count error compared to the original FOMO architecture.
- F1 Score Increase: A remarkable 225% increase in F1 score compared to the original FOMO.
- Comparison to Larger Models: The method achieved a 74% reduction in Mean Average Error (MAE) compared to YOLO, and a 85% reduction compared to YOLOv4, while fitting within the memory constraints of microcontrollers, unlike YOLO and YOLOv4.
- On-Device Performance: Experiments on the Sony Spresense platform showed that increasing the target NBA (more tiles) and input resolution led to improved F1 metrics, with a clear trade-off between latency and performance. For instance, reaching a higher target NBA of 0.008 required 3.66 times more tiles than 0.002, resulting in a substantial latency increase.
Limitations and Future Work
- Latency Trade-off: The paper acknowledges that the tiling approach introduces additional latency. While adaptive tiling helps, there's still a direct correlation between the number of tiles (and thus performance) and processing time.
- Future Work: The authors suggest that their approach is a significant step towards enabling small object detection on embedded devices. Further research could explore more sophisticated tiling strategies or integration with other lightweight detection architectures. The paper also highlights the potential for using GPS data or other sensors to estimate object size for even more precise adaptive tiling in real-world applications.
Summary updated