Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection

Xinhao Zhong, Siyu Jiao, Yao Zhao, Yunchao Wei

Abstract

Semi-supervised object detection (SSOD) has traditionally operated under a "closed-set" assumption, where all unlabeled data is presumed to belong to categories present in the labeled training set. In real-world applications, however, unlabeled data frequently contains out-of-distribution (OOD) objects. Standard SSOD models often misclassify these unknown objects into known categories, causing semantic confusion and degrading performance. This paper introduces the Collaborative Feature-Logits Detector (CFL-Detector), a framework designed to address the open-set problem by integrating feature-level contrastive clustering with logits-level uncertainty modeling. By treating "unknown" as a functional category rather than noise to be filtered, the CFL-Detector achieves state-of-the-art performance in identifying unknown objects while simultaneously improving the accuracy of known-class detection.

Introduction

While SSOD effectively leverages small labeled datasets alongside large unlabeled datasets, the presence of "open-set" data—containing both In-Distribution (ID) and Out-of-Distribution (OOD) classes—poses a significant challenge. When a model trained on "dogs" and "cats" encounters an unlabeled "horse," standard detectors typically force the horse into a known category. Existing solutions often attempt to filter out these OOD samples entirely, which is computationally expensive and fails to utilize the structural information these samples provide about the boundaries of known classes.

The CFL-Detector shifts this paradigm by learning to distinguish "known" from "unknown" through a dual-level collaborative approach. This allows the model to maintain high precision for ID objects while successfully flagging OOD objects, preventing them from poisoning the learning process.

Methodology

The CFL-Detector utilizes a Teacher-Student framework, building upon the Unbiased Teacher architecture. The core innovation lies in two collaborative loss functions that operate at different levels of the network hierarchy:

1. Feature Contrastive Loss ($\mathcal{L}_{fc}$)

At the feature level, the model employs a memory pool to store high-quality embeddings of various classes. Contrastive learning is applied to ensure that embeddings of the same class are clustered together, while embeddings of different classes—including a designated "unknown" class—are pushed apart. To maintain the integrity of the memory pool, the system uses an Intersection over Union (IoU) and cosine similarity filter, ensuring only representative, high-quality embeddings are used for contrastive refinement.

2. Uncertainty Classification Loss ($\mathcal{L}_{uc}$)

At the logits or prediction level, the model identifies potential OOD objects by analyzing background class probabilities. If an object is not classified as background but the model exhibits high uncertainty regarding its specific ID class, it is treated as a candidate for the "unknown" category. This loss function is weighted by uncertainty; the model focuses its learning efforts on ambiguous cases, forcing the network to develop sharper boundaries between known and unknown categories.

Experimental Evaluation

The researchers evaluated the CFL-Detector across three distinct experimental benchmarks designed to simulate various open-set conditions:

  • COCO-Open-CLS: Utilizes MS-COCO with 20, 40, or 60 classes designated as ID and the remainder as OOD.
  • COCO-Open-SUP: Uses a limited set of labeled ID images (1k, 2k, 4k) with the rest of the MS-COCO dataset serving as unlabeled open-set data.
  • VOC-COCO: A cross-dataset benchmark where Pascal-VOC (20 classes) serves as the labeled ID data, and the MS-COCO dataset (containing 60 non-VOC classes) serves as the unlabeled data.

Key Results

The CFL-Detector demonstrated significant improvements over existing baselines:

  • OOD Detection Capabilities: In the COCO-Open-CLS task (20 ID classes), the CFL-Detector achieved an Average Precision for unknown objects ($AP_u$) of 5.56%. In contrast, standard SSOD methods like Unbiased Teacher (UT) recorded an $AP_u$ of 0%, as they lack the mechanism to identify unknowns.
  • Improved ID Accuracy: The method also enhances performance on known categories. On the VOC-COCO benchmark, it reached 31.49 mAP, outperforming the previous state-of-the-art (DCO OSSOD) by 1.2 points.
  • Data Efficiency: The framework showed robust gains even with as few as 1,000 labeled images, highlighting its effectiveness in data-scarce environments.

Conclusion and Future Work

The CFL-Detector successfully addresses the open-set SSOD problem by moving from a strategy of OOD filtering to one of OOD learning. While the approach significantly improves unknown object detection, the authors noted a minor trade-off: a slight margin decline in ID class performance compared to models optimized strictly for closed-set environments. Future research will focus on balancing this trade-off and extending the framework to "open-world" scenarios where the variety of unknown classes is dynamic and ever-increasing.

Summary updated

BibTeX
@inproceedings{zhong2024collaborative,
  author = {Xinhao Zhong and Siyu Jiao and Yao Zhao and Yunchao Wei},
  title = {{Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection}},
  booktitle = {MMAsia},
  year = {2024},
  doi = {10.1145/3696409.3700217},
}
Figure