Faster OreFSDet : A Lightweight and Effective Few-shot Object Detector for Ore Images

Yang Zhang, Le Cheng, Yuting Peng, Chengming Xu, Yanwei Fu, Bo Wu, Guodong Sun

TL;DR

Faster OreFSDet is a lightweight, real-time few-shot object detector designed specifically for measuring ore particle sizes in industrial environments. By replacing heavy, traditional anchor-based frameworks with a streamlined CenterNet2 architecture and introducing novel feature-mining blocks, it achieves state-of-the-art accuracy at 50 FPS with a model size of just 19MB.

What Problem Does This Paper Address?

In mining and mineral processing, measuring ore particle size is critical for optimizing the crushing process and maximizing productivity. However, automating this with computer vision is notoriously difficult. Ore images suffer from dense stacking, severe occlusion, and tricky lighting reflections.

While deep learning-based object detectors can help, they require massive amounts of high-quality labeled data, which is expensive and time-consuming to collect. Few-Shot Object Detection (FSOD) algorithms can learn from just a few examples, but existing models are typically built on heavy, two-stage architectures like Faster R-CNN. As shown in Figure 1, these models generate an excessive number of region proposals, making them too slow and memory-intensive for real-world deployment on resource-constrained edge devices.

Figure 1: Faster R-CNN is usually served as the basic detector of FSOD. This anchor-based few-shot object detector uses RPN to maximize the recall of the top 1K proposals and does not use these proposal scores in the test phase. A large number of proposals slows the speed. In addition, a large number of complex modules are designed to establish the correlation between support and query, which leads to slow detection speed and high memory requirements. It is extremely challenging for scenario deployments with limited computing resources and tight memory budgets.

Methods

The authors introduce Faster OreFSDet, a lightweight and efficient FSOD framework optimized for edge deployment. The overall architecture is illustrated in Figure 2.

Figure 2: The overall framework of our proposed method. The training input of each episode consists of a query image and several support images from one class. The shared feature extractor and feature pyramid networks (FPN) first extract the query and support features. The P3, P4, and P5 of support features from FPN are fed into the SM Block for effective feature representation and then fed into the RG Block to generate attention maps with the same level query features from FPN. These attention feature maps are sent to the one-stage detector CenterNet. After filtering out the negative objects that do not belong to the support category, accurate candidate proposals are generated. Subsequently, the candidate proposals and the support features are sent to the detection head, where a more accurate parsing is conducted between the support box and the potential box in the query feature maps.

Key innovations include:

  • CenterNet2 Base Framework: Instead of using Faster R-CNN, the authors adopt CenterNet2 with a lightweight VoVNet backbone. This allows the model to use significantly fewer region proposals (256 vs. 1,000) while maintaining high proposal accuracy. The detection head is also customized to a single head with reduced channels (128) to suit single-class ore detection.
  • Support Feature Mining Block (SM Block): Traditional FSOD methods often introduce background noise from support images. The SM Block uses linear projection to encode feature representations along the height and width dimensions independently, capturing position-sensitive information and filtering out background interference.
  • Relationship Guidance Block (RG Block): This block establishes spatial and channel correlations between the support (example) and query (target) features. It pools support features into multiple scales (1x1 and 3x3) and utilizes deep strip convolutions to capture the unique, banded characteristics of ore particles.
  • Dual-Scale Semantic Aggregation Module (DSA): To handle variations in ore sizes, the DSA module extracts and aggregates features at multiple resolutions (specifically 4 and 8) during the final classification and bounding box regression stage.

Datasets

The model was evaluated using a two-phase training approach:

  1. MS COCO 2017: A massive, general-purpose dataset containing 80 target categories used for base training to help the network learn generalized feature representations.
  2. Ore Image Dataset: A newly collected dataset featuring ore images with varying distribution densities and sizes. The dataset includes 4,060 images for general object detection comparison, a validation set of 1,060 images, and few-shot subsets (5, 15, and 25 shots) used for fine-tuning the FSOD model.

Key Insights and Ideas

  • Ditching the Anchor-Based Bottleneck: Traditional FSOD models waste massive computational power by evaluating 1,000 proposals in a weak Region Proposal Network (RPN). Switching to a probabilistic first-stage detector (CenterNet2) drastically slashes the proposal count to 256 without sacrificing recall.
  • The Danger of Global Pooling: Standard few-shot techniques rely on global average pooling to extract a "prototype" vector of the target class. However, for highly irregular and overlapping objects like ores, global pooling discards vital local context and spatial details.
  • Tailoring Multi-Scale Resolution: While general object detectors often need very fine resolutions to find tiny objects, ore particles in industrial conveyor settings are predominantly medium to large. Parallel pooling at resolutions of 4 and 8 provides the optimal balance of speed and scale-awareness.

Key Results

Faster OreFSDet was evaluated against both general object detectors and state-of-the-art few-shot detectors under 5-shot, 15-shot, and 25-shot settings.

  • Accuracy: Under a 25-shot setting, Faster OreFSDet achieved a box Average Precision ($\text{AP}^{\text{box}}$) of 54.1, outperforming the AttentionRPN baseline by an incredible 23.3 points. It even beat fully-trained general object detectors like YOLOF ($\text{AP}^{\text{box}}$ of 46.8), which require thousands of training images.
  • Efficiency: The model size is compressed to just 19MB—more than 7x smaller than the ResNet50 baseline.
  • Speed: It achieves a real-time inference speed of 50 FPS on a standard GPU, making it highly competitive with general object detectors and fully viable for industrial edge deployment.

Limitations and Future Work

The authors note that this work focuses strictly on bounding-box object detection. For future work, they plan to extend this lightweight framework to more challenging tasks, such as few-shot instance segmentation (to get precise pixel-level boundaries of the ores) and one-shot object detection (learning from a single sample).

Summary updated

Abstract

For the ore particle size detection, obtaining a sizable amount of high-quality ore labeled data is time-consuming and expensive. General object detection methods often suffer from severe over-fitting with scarce labeled data. Despite their ability to eliminate over-fitting, existing few-shot object detectors encounter drawbacks such as slow detection speed and high memory requirements, making them difficult to implement in a real-world deployment scenario. To this end, we propose a lightweight and effective few-shot detector to achieve competitive performance with general object detection with only a few samples for ore images. First, the proposed support feature mining block characterizes the importance of location information in support features. Next, the relationship guidance block makes full use of support features to guide the generation of accurate candidate proposals. Finally, the dual-scale semantic aggregation module retrieves detailed features at different resolutions to contribute with the prediction process. Experimental results show that our method consistently exceeds the few-shot detectors with an excellent performance gap on all metrics. Moreover, our method achieves the smallest model size of 19MB as well as being competitive at 50 FPS detection speed compared with general object detectors. The source code is available at https://github.com/MVME-HBUT/Faster-OreFSDet.
BibTeX
@article{zhang2023faster,
  author = {Yang Zhang and Le Cheng and Yuting Peng and Chengming Xu and Yanwei Fu and Bo Wu and Guodong Sun},
  title = {{Faster OreFSDet : A Lightweight and Effective Few-shot Object Detector for Ore Images}},
  journal = {Pattern Recognition},
  year = {2023},
  url = {https://openreview.net/forum?id=cncFO1hdK8},
}
Figure