Abstract
Simultaneous localisation and categorization of objects in medical images, also referred to as medical object detection, is of high clinical relevance because diagnostic decisions often depend on rating of objects rather than e.g. pixels. For this task, the cumbersome and iterative process of method configuration constitutes a major research bottleneck. Recently, nnU-Net has tackled this challenge for the task of image segmentation with great success. Following nnU-Net's agenda, in this work we systematize and automate the configuration process for medical object detection. The resulting self-configuring method, nnDetection, adapts itself without any manual intervention to arbitrary medical detection problems while achieving results en par with or superior to the state-of-the-art. We demonstrate the effectiveness of nnDetection on two public benchmarks, ADAM and LUNA16, and propose 11 further medical object detection tasks on public data sets for comprehensive method evaluation.
Introduction
In the field of medical AI, image segmentation (pixel-level labeling) has long been the dominant paradigm. However, for clinicians, identifying and rating a tumor as a single discrete object is often more critical than precise boundary delineation. This is the domain of medical object detection, a task that is notoriously difficult to implement due to a significant "configuration bottleneck."
Standard detection models require tuning a vast array of hyperparameters, including anchor box sizes, sampling strategies for "hard negatives," and Non-Maximum Suppression (NMS) thresholds. Because medical images vary wildly—from high-resolution CT scans to lower-resolution MRIs—a configuration optimized for one anatomy rarely generalizes to another. This complexity often prevents researchers from utilizing dedicated object detection frameworks.
The nnDetection Framework
To address these challenges, we introduce nnDetection, a self-configuring framework that automatically adapts its architecture and training parameters to any medical object detection dataset. Inspired by the success of nnU-Net in segmentation, the nnDetection pipeline automates the design process through four main stages:
- Data Fingerprinting: The system analyzes the dataset to extract properties like image spacing, object sizes, and intensity distributions.
- Rule-based Parameters: Using the fingerprint, it applies heuristic rules to determine network topology, patch size, and anchor box configurations. For example, if objects are too large for a standard patch, it automatically triggers a low-resolution model to capture more context.
- Fixed Parameters: It utilizes a robust "Retina U-Net" base architecture, combining the detection heads of RetinaNet with the powerful encoder of a U-Net.
- Empirical Optimization: After training, it automatically tunes post-processing settings (like NMS thresholds) on the validation set to maximize performance.
Experimental Evaluation
We evaluated nnDetection across a massive benchmark of 12 datasets, including 11 new medical object detection tasks proposed by converting existing segmentation challenge data into detection format (e.g., RibFrac, KiTS19, and CADA).
Key Results
nnDetection demonstrates that automation does not require sacrificing performance:
- LUNA16 Benchmark: It achieved a Competition Performance Metric (CPM) of 0.930, outperforming previous methods that do not use specialized, task-specific "False Positive Reduction" stages.
- ADAM Challenge: The framework currently ranks 3rd on the live leaderboard for aneurysm detection.
- Broad Superiority: In head-to-head comparisons, nnDetection consistently outperformed optimized segmentation baselines (nnU-Net), particularly in tasks with small or complex objects.
Discussion and Conclusion
A central insight of this work is that dedicated detection models, when properly configured, are superior to segmentation-based approaches for object-level tasks. By automating the optimization of anchor boxes—the traditional "Achilles' heel" of detection—via differential evolution, nnDetection removes the primary barrier to entry for medical researchers.
While nnDetection provides a high-performance baseline, it is intended as a robust starting point rather than a final solution for every specialized task. It does not currently include task-specific False Positive Reduction (FPR) modules often used in specialized lung nodule detectors. However, its ability to "just work" across arbitrary datasets represents a significant step toward reproducible and generalizable medical AI.
Code and models are available at: https://github.com/MIC-DKFZ/nnDetection
Summary updated