AutoSelect: Automatic and Dynamic Detection Selection for 3D Multi-Object Tracking

Xinshuo Weng, Kris KitaniarXiv 2020 · arXiv

Abstract

3D Multi-Object Tracking (MOT) typically relies on manual, static confidence thresholds to filter false positive detections. This paper presents AutoSelect, an end-to-end trainable module that automatically learns dynamic selection criteria at both frame and instance levels. By integrating detection selection directly into the tracking pipeline, AutoSelect eliminates the need for exhaustive manual tuning and adapts to varying environmental contexts. Experimental results on the KITTI and nuScenes benchmarks demonstrate that AutoSelect significantly reduces false positives and achieves state-of-the-art performance.

Introduction

Modern 3D MOT systems generally follow a "tracking-by-detection" pipeline. Because real-world 3D object detectors inevitably produce false positives (FPs), trackers must filter incoming detections using a confidence score threshold before data association. Historically, finding this threshold required exhaustive manual search via ablation studies.

Comparison of traditional manual thresholding versus the proposed AutoSelect framework.

However, a single global threshold is sub-optimal; it fails to account for variations across different frames or specific object contexts. For instance, a distant true positive (TP) might have a lower confidence score than a nearby false positive. AutoSelect addresses this by replacing rigid pre-processing heuristics with an automated, dynamic selection mechanism that adapts on the fly.

Methods

The authors build their system on a Graph Neural Network (GNN)-based 3D MOT architecture. The core innovation is the AutoSelect module, which is integrated into the pipeline in two distinct variants:

  1. Frame-level AutoSelect (TRN): A Threshold Regression Network (TRN) utilizes multi-modal features (2D/3D motion and appearance) from past tracklets and current detections to predict a dynamic threshold $\hat{\tau}$ for the entire frame. Detections scoring below this threshold are discarded.
  2. Instance-level AutoSelect: This variant predicts a "true positiveness" probability for each individual detection using a shared Multi-Layer Perceptron (MLP). This allows the system to simultaneously retain low-score TPs and discard high-score FPs within the same frame based on local context.
Detailed architecture of the end-to-end 3D MOT system featuring AutoSelect for detection selection and a GNN for data association.

Unlike prior heuristic-based methods, AutoSelect is optimized end-to-end alongside the GNN-based data association module. The training process employs a binary cross-entropy loss for selection probability combined with triplet and affinity losses for tracking associations.

Key Insights

  • Context-Aware Filtering: By incorporating features from already tracked objects, the network leverages historical context to better identify which new detections are likely false positives.
  • Instance-level Superiority: Predicting actual "true positiveness" per detection is more effective than a frame-wide threshold. The network learns that a raw confidence score is merely an approximation of quality, allowing for more nuanced filtering.
  • End-to-End Optimization: Moving detection filtering from a hardcoded pre-processing step to a differentiable module allows the selection criteria to directly serve the final tracking objective.

Results and Discussion

The proposed method was evaluated on the KITTI and nuScenes datasets, showing significant improvements over traditional baselines:

  • False Positive Reduction: On KITTI, AutoSelect filtered out 45.7% of false positives while maintaining high recall.
  • State-of-the-Art Performance: At a strict 3D IoU threshold of 0.5, Instance-level AutoSelect achieved a sAMOTA of 92.85% on KITTI. On the more complex nuScenes dataset, it reached a sAMOTA of 48.93%, significantly outperforming the GNN3DMOT baseline (29.84%).

Conclusion and Future Work

AutoSelect demonstrates that automating the detection selection process not only removes the burden of manual tuning but also enhances tracking robustness through dynamic adaptation. While currently tailored for 3D MOT using LiDAR and point cloud features, future work could involve extending this methodology to 2D MOT benchmarks to evaluate its effectiveness in camera-only, video-based tracking systems.

Summary updated

Abstract

3D multi-object tracking is an important component in robotic perception systems such as self-driving vehicles. Recent work follows a tracking-by-detection pipeline, which aims to match past tracklets with detections in the current frame. To avoid matching with false positive detections, prior work filters out detections with low confidence scores via a threshold. However, finding a proper threshold is non-trivial, which requires extensive manual search via ablation study. Also, this threshold is sensitive to many factors such as target object category so we need to re-search the threshold if these factors change. To ease this process, we propose to automatically select high-quality detections and remove the efforts needed for manual threshold search. Also, prior work often uses a single threshold per data sequence, which is sub-optimal in particular frames or for certain objects. Instead, we dynamically search threshold per frame or per object to further boost performance. Through experiments on KITTI and nuScenes, our method can filter out $45.7\%$ false positives while maintaining the recall, achieving new S.O.T.A. performance and removing the need for manually threshold tuning.
BibTeX
@misc{weng2020autoselect,
  author = {Xinshuo Weng and Kris Kitani},
  title = {{AutoSelect: Automatic and Dynamic Detection Selection for 3D Multi-Object Tracking}},
  year = {2020},
  eprint = {2012.05894},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2012.05894},
}
Figure