Object detectors attach a confidence score to every box they predict. Downstream systems treat that score as a probability, for example when a drone counts vehicles or an inspection line flags defects. For tiny objects only a few pixels wide, these scores are among the least trustworthy a detector produces. Part of the reason is real: the network shrinks its feature maps step by step (downsampling), and that compression erases faint signals. Part of it may be a measurement artifact. Detection correctness is usually defined by box overlap (Intersection over Union, IoU), which treats a box that is off by two pixels on an 8-pixel object as a miss. Calibration losses inherit that definition. So do the expected calibration error (ECE) metrics used to judge them, which are binned and non-differentiable. Those losses therefore push tiny-object confidences down for detections that are, by any practical standard, correct.
We propose a train-time calibration loss with two properties no current loss has. First, it defines correctness with a similarity measure that stays smooth and scale-invariant for tiny boxes, instead of IoU. Second, it conditions calibration on object scale, so that a small batch of tiny detections is not swamped by large ones. We test it in the regime where post-hoc fixes should break, which is a change in effective resolution between calibration time and deployment. We also use it as a tool to separate how much tiny-object miscalibration comes from the IoU artifact and how much comes from information lost during downsampling.
Core Claim
Existing differentiable detection calibration losses all compute correctness through IoU and pool every scale into one objective. These are the kernel density estimator of Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detection, the soft precision–confidence quadrants of Bridging Precision and Confidence (BPC), and the localization term of Multiclass Confidence and Localization Calibration (MCCL).
Detecting tiny objects in aerial images shows the problem with IoU at this scale. A 3-pixel shift on a 5×8 box drops IoU from 0.54 to 0.14. That paper also proposes the Normalized Wasserstein Distance (NWD), which models each box as a 2D Gaussian and gives a smooth similarity even when boxes do not overlap. Multivariate Confidence Calibration for Object Detection shows that miscalibration depends strongly on box width and height, but it corrects this only after training, on frozen models.
Our claim is that a loss which estimates calibration conditionally on scale, with NWD-based soft correctness, will do three things. It will reduce tiny-object calibration error measured under both IoU and NWD correctness. It will cost no accuracy on very tiny objects. And it will stay calibrated when test imagery is coarser than the training imagery, which is where scale-aware post-hoc calibrators fitted at native resolution should fail.
What is new is not "use NWD" and not "use KDE" on their own. It is the combination of three things: a scale-conditional estimator built for sparse tiny detections, trained end-to-end, and evaluated under resolution shift.
Research Program
Let $p$ be a detection's confidence and $s=\log\sqrt{wh}$ its log box size. Let $c\in[0,1]$ be its soft correctness, defined as the NWD to the best-matching ground-truth box of the same class, passed through a link function in the sense of Popordanoska et al. The quantity we target is scale-conditional calibration error:
$$\mathrm{CE}_{s}=\mathbb{E}_{p,s}\big[\,\lvert \mathbb{E}[c\mid p,s]-p\rvert\,\big],$$
which is zero only if the detector is calibrated at every size simultaneously.
We estimate the inner expectation with a Nadaraya–Watson kernel average over pairs of detections $(i,j)$:
$$\hat{\mathbb{E}}[c\mid p_i,s_i]=\frac{\sum_{j\ne i}K_h(p_i-p_j)\,K_g(s_i-s_j)\,c_j}{\sum_{j\ne i}K_h(p_i-p_j)\,K_g(s_i-s_j)}.$$
Here $K_h$ and $K_g$ are Gaussian kernels with bandwidths $h$ (over confidence) and $g$ (over scale). The training loss is the detection loss plus $\lambda$ times the empirical $\mathrm{CE}_s$.
This estimator is differentiable in $p$, so it avoids the zero-gradient problem of binned ECE. The scale kernel keeps tiny detections from being averaged against large ones.
Sparsity is the obstacle that Average Calibration Error did not face. That work could use hard bins because segmentation supplies millions of voxels per image, and tiny-object detection supplies a handful of boxes. We handle this with a first-in-first-out queue of detached past detections, so the conditional average has enough neighbors. Gradients flow only through the current batch's confidences. The pairwise cost is quadratic in queue size, so we cap the queue and report wall-clock overhead.
Bandwidths are known to bias kernel estimates (Towards Unbiased Calibration using Meta-Regularization). We therefore fix them by a leave-one-out rule on training data and report how sensitive results are to that choice. We do not tune them on test calibration.
The second component is a diagnostic decomposition. For a fixed detector, we compute tiny-object calibration error twice, once under IoU-at-0.5 correctness and once under NWD correctness. The gap measures how much apparent miscalibration is an artifact of the correctness definition. We then swap the backbone's strided downsampling for LIP. LIP is learnable importance-weighted pooling that raised COCO small-object AP from 21.9% to 24.0% but was never evaluated for calibration. Any reduction in NWD-based calibration error that LIP produces under plain training is evidence that some miscalibration comes from information erased by downsampling.
As a secondary question, we ask whether letting the calibration gradient reach LIP's importance module changes what the pooling keeps. We frame this as exploratory. The main claim does not depend on it.
Baselines
The naive reference point is the detector trained with NWD and ranking-based assignment, with no calibration treatment. That paper reports 24.7 AP on AI-TOD-v2 with DetectoRS. It reports no calibration figure, and we found no reported calibration result on Lacuna for AI-TOD-v2 at all. We therefore first establish scale-stratified calibration error for this baseline ourselves. Every other method starts from the same checkpoint, schedule, and data.
The strongest train-time competitor is the KDE calibration loss of Popordanoska et al. It is the closest method: kernel-based, differentiable, and statistically consistent. It reports cutting COCO calibration error from 21.89 to 9.72 with RetinaNet. BPC is the second train-time competitor and reports D-ECE dropping from 12.8% to 10.3% on MS-COCO. Both are run with their IoU-based correctness as published. Both get the same fine-tuning budget as our loss and an equal hyperparameter grid for their weight term, selected on a held-out training split.
The strongest post-hoc competitor is the dependent multivariate calibration of Küppers et al. It takes confidence, box center, and box size as inputs, and is fitted on a validation split. No reported number for it was found on Lacuna in the file, so we re-implement it. We also include standard temperature scaling and temperature scaling fitted separately per size bucket.
Two ablations isolate the key ingredients. The first is the KDE loss with NWD correctness but no scale kernel ($g\to\infty$), which isolates conditioning. The second is our loss with IoU correctness, which isolates the similarity measure.
Beating the baselines means three things. We need lower calibration error on very-tiny and tiny objects under both correctness definitions. We need AP on very-tiny objects within noise of the uncalibrated baseline. And under resolution shift we need a clear margin over the post-hoc calibrators, over three seeds.
Why not just…?
Why not just fit Küppers-style multivariate calibration on box size after training? In-domain, this is the real threat. A post-hoc map conditioned on width and height may match our loss when the validation and test resolutions agree, and we will report that honestly if it happens. Its weak point is that it learns a mapping from pixel size to correction. When imagery is captured from higher altitude, a car that was 16 pixels wide becomes 8 pixels wide. The map then applies the correction learned for genuinely small classes rather than the one for degraded large ones. Our resolution-shift experiment tests exactly this. If the post-hoc map holds up there too, the end-to-end claim fails, and the contribution reduces to the diagnostic plus a cheaper recipe.
Why not just put NWD inside the existing KDE or BPC loss? This is our no-scale-kernel ablation. With a pooled kernel estimate over confidence alone, tiny detections share neighbors with large, confident ones. This repeats the scale-dependence failure Küppers et al. documented for univariate methods. If the pooled NWD variant matches ours, the scale conditioning is unnecessary and we will say so.
Why not just preserve resolution with LIP or higher-resolution feature maps and skip the calibration loss? LIP's gains are measured only in AP, and better features do not imply calibrated scores. Moreover, under IoU correctness, a more accurate tiny-object detector is still judged against an unstable target. The LIP-only arm of the decomposition answers this directly.
Why not just use temperature scaling? Prior evidence in the file argues against it. Popordanoska et al. and BPC both report train-time losses beating temperature scaling. BPC reports Foggy Cityscapes D-ECE of 12.5% versus 19.5% for the baseline. A single temperature also cannot change the relative miscalibration between sizes. We include it anyway, including the per-bucket variant.
Proposed Experiments
The main benchmark is AI-TOD-v2, which has a mean object size of 12.7 pixels. We use a Faster R-CNN detector with NWD assignment, and DetectoRS if compute allows. Results are stratified by AI-TOD-v2's very-tiny, tiny, small, and medium bins.
We report four calibration measures per bin: D-ECE under IoU-at-0.5 correctness, D-ECE under NWD-based correctness, the KDE estimate of conditional calibration error, and reliability diagrams. Accuracy is reported as AP and very-tiny AP. Reporting both correctness definitions guards against the objection that we redefined success to suit our loss.
The decisive check is resolution shift. Every model and post-hoc calibrator is fitted at native resolution, then evaluated on test images downsampled by factors of 1.5 and 2. This moves objects into smaller size bins while their semantic identity stays the same. A cross-dataset transfer to VisDrone2019 adds a second, natural shift.
To show the method does not harm ordinary scales, we also report COCO results. These are stratified by small, medium, and large objects, using RetinaNet so the numbers are comparable with the 9.72 figure from Popordanoska et al.
Three further analyses complete the study. The decomposition experiment runs standard versus LIP backbones under plain and calibrated training. We also sweep queue size and bandwidth to locate where sparsity breaks the estimator. Finally, we measure training overhead against the KDE and BPC baselines.
The proposal is supported if three results hold. Scale-conditioned NWD calibration must beat pooled IoU-based KDE and BPC on tiny-object calibration error. It must beat multivariate post-hoc calibration under resolution shift. And very-tiny AP must stay within noise.
References
- Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detection
- Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection
- Multiclass Confidence and Localization Calibration for Object Detection
- Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark
- Multivariate Confidence Calibration for Object Detection
- Average Calibration Error: A Differentiable Loss for Improved Reliability in Image Segmentation
- Towards Unbiased Calibration using Meta-Regularization
- LIP: Local Importance-Based Pooling