On the Importance of Large Objects in CNN Based Object Detection Algorithms

Ahmed Ben Saad, Gabriele Facciolo, Axel DavyWACV 2024

TL;DR

This paper argues that training object detection models with a focus on large objects, even if they are less frequent, leads to better performance across all object sizes. They propose a simple yet effective loss re-weighting scheme that prioritizes larger objects during training, demonstrating significant improvements in mAP and MAE across various detectors and datasets.

What Problem Does This Paper Address?

Object detection models are fundamental to computer vision, powering applications from autonomous driving to surveillance. While significant progress has been made, achieving consistently high performance across all object sizes remains a challenge. Existing methods often struggle with small objects due to their limited visual information and reliance on context, while large objects can also present unique detection difficulties. The paper highlights that the distribution of object sizes in training datasets might not be optimal for learning robust features that generalize well to all scales. They question the common intuition that focusing training on the most frequent object sizes (often small to medium) is always the best strategy.

Methods

The core contribution of this paper is a novel loss re-weighting scheme designed to emphasize large objects during the training of CNN-based object detectors. Instead of treating all objects equally in the loss function, they introduce a weighting term, $W_i = \log(h_i \times w_i)$, where $h_i$ and $w_i$ are the height and width of the $i$-th object. This term is incorporated into both the classification and localization losses. The logarithm ensures that while larger objects receive more weight, no object size is entirely neglected. This approach aims to encourage the model to learn more discriminative and generalizable features from larger objects, which can then benefit the detection of smaller objects as well. They also explore the impact of pre-training on large objects before fine-tuning on the entire dataset.

Datasets

The primary datasets used in this paper are:

  • COCO (Common Objects in Context): Used extensively for training and evaluation, including ablation studies and experiments with various object detectors.
  • NuScenes: Used to validate the generalizability of the proposed weighting scheme on a different dataset.

Key Insights and Ideas

  • Large objects are feature-rich teachers: The paper posits that large objects, due to their inherent detail and texture, provide richer learning signals. Focusing on these signals can lead to the learning of more robust and generalizable features that benefit the detection of all object sizes.
  • Counter-intuitive training strategy: Contrary to the intuition that one should focus on the most frequent object sizes, this work demonstrates that prioritizing larger, less frequent objects can lead to superior overall performance.
  • Simple yet effective modification: The proposed loss re-weighting is a straightforward modification to existing object detection frameworks, requiring minimal architectural changes and offering significant performance gains.
  • Impact on low-level features: Ablation studies suggest that the weighting scheme primarily impacts low-level features in the network, suggesting that these generic features learned from large objects are more transferable to smaller objects.
  • Dataset bias correction: The weighting scheme effectively addresses inherent dataset biases where smaller objects are more numerous but harder to detect, by artificially increasing the influence of larger objects.

Key Results

The paper reports significant performance improvements across various object detectors (YOLO v5, InternImage-T, DETR, Mask R-CNN) and datasets (COCO, NuScenes) when using the proposed weighting scheme.

  • InternImage-T on COCO: Achieved a 4 p.p. gain in mAP (from 47.2% to 51.2%) with the proposed changes.
  • Pre-training on large objects: Models pre-trained on large objects and then fine-tuned on the entire dataset consistently outperformed models pre-trained on small/medium objects, even for small object detection.
  • Ablation studies: Applying the weighting to both classification and detection losses yielded the best results, showing substantial improvements in mAP (up to 12 p.p. for small objects) and MAE (up to 43% decrease for small objects) compared to the initial results.
  • Qualitative improvements: Figure 5 shows examples where objects that were previously undetected are now successfully identified with the proposed weighting. Bounding box predictions also became more precise.

Limitations and Future Work

  • Dataset dependency: The authors acknowledge that the effectiveness of the weighting scheme might depend on the specific object size distribution and the inherent difficulty of detecting objects of different sizes within a dataset. Datasets with already optimal proportions might see less benefit.
  • Optimal weighting function: While $\log(w \times h)$ proved effective, the paper suggests that further research is needed to identify potentially better weighting functions or to definitively prove the optimality of the chosen one.
  • Future research directions: The authors suggest investigating novel strategies that explicitly consider the impact of large objects on detection accuracy across different scales. They also propose exploring other datasets and object detectors to further validate the approach.

Summary updated

BibTeX
@inproceedings{saad2024importance,
  author = {Ahmed Ben Saad and Gabriele Facciolo and Axel Davy},
  title = {{On the Importance of Large Objects in CNN Based Object Detection Algorithms}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure