CoIn: Contrastive Instance Feature Mining for Outdoor 3D Object Detection with Very Limited Annotations

Qiming Xia, Jinhao Deng, Chenglu Wen, Hai Wu, Shaoshuai Shi, Xin Li, Cheng WangICCV 2023 · DOI

Abstract

Training high-performing 3D object detectors typically requires massive amounts of expensive 3D bounding box annotations. This paper introduces CoIn (Contrastive Instance feature mining), a framework designed to bridge the performance gap between sparsely annotated and fully supervised models. By leveraging multi-class contrastive learning and mining "latent" instances directly in the feature space, CoIn enables effective training with as little as 2% of standard labels.

Introduction

3D object detection is a cornerstone of autonomous driving and robotics, yet it faces a significant "data hunger" problem. Annotating 3D bounding boxes in point clouds is labor-intensive and costly. When state-of-the-art (SOTA) detectors are trained with very few labels (e.g., 1-2%), performance typically plummets due to two main factors:

  1. Indistinguishable Features: Limited supervision makes it difficult for models to differentiate between foreground objects (cars, pedestrians) and background clutter.
  2. Unreliable Pseudo-labels: Standard semi-supervised methods rely on "teacher" models whose predictions are often too noisy in low-data regimes to provide useful guidance.

The CoIn Framework

To address these challenges, the authors propose CoIn, which shifts the focus from traditional bounding box prediction to robust feature-level mining. Unlike traditional methods that mine pseudo-labels in the "box space," CoIn argues that mining in the "feature space" is more robust when labels are extremely scarce.

The CoIn architecture integrates multi-class contrastive learning and instance feature mining to optimize 3D detection with sparse labels.

The framework consists of three core components:

  • Multi-Class Contrastive Learning (MCcont): This module forces the model to compare features across all categories simultaneously, pulling same-class features together and pushing different classes apart to improve discriminative power.
  • Instance Feature Mining (InF-Mining): Instead of relying on noisy box predictions, this module identifies "latent" instances in the Bird’s-Eye-View (BEV) feature map by calculating similarity to labeled "meta-instances."
  • Labeled-to-Pseudo Contrastive Learning (LPcont): This acts as a quality control mechanism, using high-quality labeled features to supervise and refine the mined pseudo-features.

The iterative CoIn++ pipeline further combines these modules with self-training to refine results over multiple stages.

Experimental Results

CoIn was evaluated on major benchmarks including KITTI, Waymo Open Dataset, and nuScenes. The results demonstrate that organizing the feature space via contrastive learning allows even simple detectors to perform remarkably well with minimal supervision.

Comparison of Centerpoint vs. CoIn across various annotation rates on the KITTI dataset. CoIn maintains high performance even at 2% annotation.

Key findings include:

  • Efficiency: With only 2% of annotations, CoIn++ achieves performance comparable to models trained on 100% of the data.
  • Baseline Improvements: On the KITTI "Car" category (Moderate difficulty), CoIn boosted the CenterPoint baseline from a dismal 31.55% AP to 54.82% AP—a massive 23-point jump.
  • Versatility: The method is architecture-agnostic, showing success when integrated into single-stage (CenterPoint), two-stage (Voxel-RCNN), and multi-stage (CasA) detectors.
  • Scalability: On the large-scale Waymo dataset, CoIn outperformed baselines by over 16% in AP/APH metrics for vehicles.

Conclusion

CoIn demonstrates that the primary bottleneck in low-annotation 3D detection is the organization of the feature space. By utilizing contrastive learning and feature-level mining, the framework effectively closes the gap between semi-supervised and fully supervised performance. Future work will explore adapting these techniques to long-tail scenarios and multi-modal sensor fusion.

Summary updated

BibTeX
@inproceedings{xia2023coin,
  author = {Qiming Xia and Jinhao Deng and Chenglu Wen and Hai Wu and Shaoshuai Shi and Xin Li and Cheng Wang},
  title = {{CoIn: Contrastive Instance Feature Mining for Outdoor 3D Object Detection with Very Limited Annotations}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00575},
}
Figure