Abstract
Object proposals have become an integral preprocessing steps of many vision pipelines including object detection, weakly supervised detection, object discovery, tracking, etc. Compared to the learning-free methods, learning-based proposals have become popular recently due to the growing interest in object detection. The common paradigm is to learn object proposals from data labeled with a set of object regions and their corresponding categories. However, this approach often struggles with novel objects in the open world that are absent in the training set. In this paper, we identify that the problem is that the binary classifiers in existing proposal methods tend to overfit to the training categories. Therefore, we propose a classification-free Object Localization Network (OLN) which estimates the objectness of each region purely by how well the location and shape of a region overlap with any ground-truth object (e.g., centerness and IoU). This simple strategy learns generalizable objectness and outperforms existing proposals on cross-category generalization on COCO, as well as cross-dataset evaluation on RoboNet, Object365, and EpicKitchens. Finally, we demonstrate the merit of OLN for long-tail object detection on large vocabulary dataset, LVIS, where we notice clear improvement in rare and common categories.
Introduction
Most modern computer vision pipelines rely on object proposals—candidate regions likely to contain an object. Traditionally, these are generated by a Region Proposal Network (RPN) that uses a binary classifier to distinguish "foreground" from "background." However, this approach suffers from a fundamental flaw: classification-based objectness overfits. When a model is trained only on common categories (e.g., cats, dogs, cars), the binary classifier learns to treat anything that doesn't resemble those categories as "background." Consequently, novel objects in "open-world" scenarios—such as specialized medical tools or rare machinery—are often ignored. The challenge lies in creating a class-agnostic proposer that understands the general geometric properties of an "object" without requiring explicit category knowledge.
Methodology: The Object Localization Network (OLN)
To address the limitations of classification-based proposals, we introduce the Object Localization Network (OLN). OLN replaces standard classification heads in two-stage detectors with localization quality estimators.
- OLN-RPN (Stage 1): Instead of predicting foreground/background probability, this stage predicts centerness, identifying how close a point is to the center of a potential object.
- OLN-Box (Stage 2): Suggested regions are refined by predicting an IoU (Intersection over Union) score, measuring how well the proposed box overlaps with actual object boundaries.
- Inference: The final "objectness" score is the geometric mean of the centerness and IoU scores.
By training purely on localization targets (how well a box fits any object) rather than discrimination targets (is this a member of a known class?), OLN learns a universal language of "where" objects are, rather than "what" they are.
Experimental Evaluation
We evaluated OLN across several diverse datasets to test its generalization capabilities, including COCO, Object365, and the egocentric EpicKitchens dataset. We also exhaustively annotated 109 images from the RoboNet robotics dataset to create a more rigorous open-world benchmark.
Key Results
OLN demonstrates significant improvements in open-world scenarios:
- Cross-Category Generalization: On COCO, OLN outperformed state-of-the-art methods by +3.3% AUC on unseen categories.
- Cross-Dataset Robustness: On RoboNet, OLN showed massive improvements, outperforming standard baselines by +13 to 16 points in Average Precision (AP).
- Long-Tail Success: As a drop-in replacement for RPN in LVIS, OLN improved detection of "rare" categories by +3.4 points (APr).
Conclusion
The Object Localization Network demonstrates that classification is often the bottleneck for generalization in object detection. By focusing on geometric rules like centerness and IoU, OLN avoids the "background trap" where novel objects are suppressed. While OLN occasionally struggles with over-segmentation of complex objects, it provides a robust foundation for open-world vision systems. Future work includes integrating OLN with zero-shot learning to enable naming of the novel objects it discovers.
Summary updated