Abstract: nnDetection - A Self-configuring Method for Medical Object Detection

Michael Baumgartner, Paul F. Jäger, Fabian Isensee, Klaus H. Maier-HeinBildverarbeitung für die Medizin 2022 · OpenReview

Introduction

In medical imaging, the ability to simultaneously localize and categorize structures—known as medical object detection—is of paramount clinical importance. Diagnostic decisions often hinge on identifying and rating specific objects, such as tumors or lesions, rather than analyzing individual pixels. Despite this, researchers frequently default to semantic segmentation (pixel-level labeling) because object detection pipelines are notoriously difficult to configure.

Standard detection models like RetinaNet require experts to manually tune "anchors" (pre-defined bounding box shapes), sampling strategies, and complex post-processing steps. Because medical images vary wildly—ranging from high-resolution CT scans of tiny lung nodules to MRI scans of large organs—a configuration optimized for one task rarely generalizes to another. nnDetection addresses this "researcher-in-the-loop" bottleneck by introducing a self-configuring framework that adapts its architecture and hyperparameters to the specific "fingerprint" of any medical dataset.

The Self-Configuring Methodology

nnDetection follows a systematic, four-stage recipe to build an optimized detection pipeline without manual intervention:

  1. Data Fingerprinting: The system begins by analyzing the dataset to extract critical properties, including image spacing, intensity distributions, and median object sizes.
  2. Rule-based Parameters: Using the data fingerprint, nnDetection automatically determines the network topology, patch size, and anchor box scales. Notably, it employs differential evolution—a type of optimization algorithm—to find the best-fitting anchor sizes for the specific objects in the training set.
  3. Fixed Parameters: The framework utilizes a "Retina U-Net" as its base architecture. This hybrid approach combines the robust feature extraction capabilities of a U-Net with the specialized detection heads of a RetinaNet.
  4. Empirical Optimization: Following training, the system automatically tunes inference-time hyperparameters, such as Non-Maximum Suppression (NMS) thresholds, on the validation set to maximize final performance.

Additionally, the framework includes a "Low Resolution Model" trigger. If objects are too large to fit within a standard training patch, nnDetection automatically creates a downsampled version of the pipeline to ensure the model captures sufficient global context.

Benchmarking and Datasets

The authors evaluated nnDetection across 13 different medical datasets, encompassing a wide variety of modalities and targets:

  • Public Benchmarks: LUNA16 (Lung nodules) and ADAM (Brain aneurysms).
  • Development Pool: 10 diverse datasets including CADA (Aneurysms), RibFrac (Rib fractures), and Kits19 (Kidney tumors).
  • Standardization: A major contribution of this work is the conversion of 11 existing medical segmentation challenges into standardized object detection benchmarks, providing a new foundation for community evaluation.

Key Results and Insights

The performance of nnDetection demonstrates that systematized deep learning can outperform manually tuned models:

  • State-of-the-Art Performance: On the LUNA16 benchmark, nnDetection achieved a Competition Performance Metric (CPM) of 0.930, setting a new record for models that do not utilize task-specific false positive reduction stages.
  • Leaderboard Success: The framework ranked 3rd on the ADAM live leaderboard at the time of publication, showing high sensitivity in detecting brain aneurysms.
  • Detection vs. Segmentation: In a large-scale comparison against an optimized nnU-Net baseline across 12 datasets, nnDetection was superior in 9 cases, often by margins exceeding 5% mAP. This suggests that dedicated detection models are often preferable to segmentation-based approaches when configured correctly.

Discussion and Future Work

The success of nnDetection highlights a shift in medical AI: the most significant gains often come from intelligently adapting existing architectures to data rather than designing entirely new ones. The ability to automatically optimize anchor boxes is particularly critical, as medical objects range from spherical nodules to tubular vessels.

While highly effective, the authors identify areas for further refinement. Currently a one-stage framework, nnDetection could potentially reach even higher accuracy by incorporating an automated "two-stage" refinement process to filter false positives. Additionally, while the framework saves human time, it remains computationally intensive due to the requirement of five-fold cross-validation. Future iterations may integrate Neural Architecture Search (NAS) to further refine the network blueprint stage.

Summary updated

Abstract

Simultaneous localisation and categorization of objects in medical images, also referred to as medical object detection, is of high clinical relevance because diagnostic decisions often depend on rating of objects rather than e.g. pixels. For this task, the cumbersome and iterative process of method configuration constitutes a major research bottleneck. Recently, nnU-Net has tackled this challenge for the task of image segmentation with great success.
BibTeX
@inproceedings{baumgartner2022abstract,
  author = {Michael Baumgartner and Paul F. Jäger and Fabian Isensee and Klaus H. Maier-Hein},
  title = {{Abstract: nnDetection - A Self-configuring Method for Medical Object Detection}},
  booktitle = {Bildverarbeitung für die Medizin},
  year = {2022},
  url = {https://openreview.net/forum?id=8VY5x9dSX_4},
}
Figure