Shape-Aware Confidence for Oriented Object Detectors

Detectors that find rotated objects in aerial and satellite images (ships, bridges, vehicles seen from above) now routinely represent each box as a two-dimensional Gaussian, a smooth "blob" whose center is the box center and whose spread encodes the box's width, height and rotation. This trick made training far more stable, but the Gaussian carries no information about how sure the detector is. Its spread describes the object's size, not the model's doubt. The confidence scores these detectors output were never checked against how often the boxes are actually right. We propose to have the detector predict a second covariance: its uncertainty about where the box really is, expressed in the same Gaussian space as the box itself. Because both live in that space, the detector can compute in closed form how much that uncertainty should reduce its expected overlap with the true object. A fixed positional wobble costs almost nothing on a square storage tank and is severe on a thin ship. The detector then reports a confidence that is calibrated at any overlap threshold the user picks.

The core bet is that miscalibration in oriented detection is largely a matter of shape. Oriented box errors are coupled: a 2° angle error barely matters for a square but destroys overlap for an object with aspect ratio 10, as the KLD analysis showed (Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence). If calibration error tracks that coupling, a confidence that knows the box's shape should beat one that learns shape effects from sparse validation data. We test this premise first. If it fails, the method is not worth building.

Where this sits

The Gaussian box line supplies the geometry. GWD (Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss), KLD, and KFIoU (The KFIoU Loss for Rotated Object Detection) turn a rotated box into $\mathcal{N}(\mu,\Sigma)$ and regress it with distances between distributions. KFIoU approximates rotated overlap through the product of two Gaussians. GauCho (GauCho: Gaussian Distributions with Cholesky Decomposition for Oriented Object Detection) predicts the Cholesky factor of $\Sigma$ directly, which avoids angles altogether. All four treat $\Sigma$ as deterministic geometry and never evaluate calibration. We reuse GauCho's parameterization and KFIoU's overlap algebra, and add the missing uncertainty.

The calibration line motivates the work and constrains the design. Multivariate Confidence Calibration for Object Detection showed that detector miscalibration varies with box position and size. It fixes this with post-hoc logistic or beta calibration on $(z, c_x, c_y, w, h)$, where $z$ is the logit, and it defines the detection calibration error (D-ECE) at a fixed overlap threshold. Multiclass Confidence and Localization Calibration for Object Detection (MCCL) moved calibration into training with an auxiliary loss. That loss uses Monte Carlo dropout, meaning several stochastic forward passes per batch, and the authors flag this overhead as an open problem. Both methods work only with horizontal boxes.

Closest in mechanism is Bounding Box Regression With Uncertainty for Accurate Object Detection. It predicts an independent 1D Gaussian variance for each box edge, and explicitly leaves correlated multivariate error models to future work. It does not calibrate classification scores. Our proposal is that future work, carried out in rotated geometry and tied to confidence.

Core Claim

The detector head outputs three things: the geometric Gaussian $\mathcal{N}(\mu,\Sigma)$ in Cholesky form, a class score, and a predictive uncertainty. The uncertainty has two parts. The first is a 2×2 covariance $U$ over the true center. The second is a small diagonal covariance over the three Cholesky entries, which captures uncertainty in shape and rotation. Because Cholesky space has no angle wraparound, this uncertainty can be trained with an ordinary Gaussian negative log-likelihood. That avoids the discontinuities that would affect an independent angle variance.

For the center part, the expected KFIoU center-alignment term has a closed form. Suppose the true center deviates from the predicted $\mu$ by $d \sim \mathcal{N}(0, U)$, and approximate the true shape by the predicted $\Sigma$. Then

$$ \mathbb{E}_d\left[\exp\left(-\tfrac{1}{2} d^\top (2\Sigma)^{-1} d\right)\right] = \det\left(I + (2\Sigma)^{-1} U\right)^{-1/2}, $$

where $I$ is the 2×2 identity. This is the formula that makes confidence shape-aware. The same $U$ lowers the expected overlap a little when it points along a long axis of $\Sigma$, and a lot when it points across a short axis.

The shape uncertainty is propagated through the KFIoU overlap term with an unscented transform, which uses seven sigma points evaluated only on head outputs. The network still needs just one forward pass. Together these give an expected overlap $q$ for each detection.

A small monotone link then maps $(q, t)$ to $P(\text{IoU} \ge t)$ for any threshold $t$, where IoU (intersection over union) is the standard overlap measure between boxes. The reported confidence is

$$ s(t) = p_{\text{cls}} \cdot g_\phi(q, t), $$

where $p_{\text{cls}}$ is the class probability and $g_\phi$ is the learned link. The link and the class head are trained with binary cross-entropy against whether a prediction's true rotated IoU clears $t$, with $t$ sampled in $[0.5, 0.95]$ at each step. This replaces MCCL's dropout sampling with an analytic, single-pass calibration target.

What is new is the combination of three things: calibration that is conditioned on the overlap threshold, trained at training time, and derived from a predictive covariance that lives in the same Gaussian space as the box geometry. None of the cited works does this, and all of the calibration work is limited to horizontal boxes.

Research Program

The first stage is a measurement study that addresses Gap 2 directly. We train RetinaNet and FCOS oriented detectors with Smooth L1, KLD, KFIoU and GauCho heads using the AlphaRotate codebase (yangxue0827/RotationDetection). We then measure D-ECE on DOTA-v1.0 and HRSC2016, broken down by aspect-ratio bin and orientation bin, at IoU thresholds of 0.5, 0.75 and 0.9. The go/no-go criterion is whether calibration error rises with aspect ratio at the stricter thresholds after global temperature scaling. If the curves are flat, the shape-aware mechanism has nothing to fix, and we would report the audit as the contribution.

The second stage builds the head described above and derives the unscented propagation for the KFIoU overlap term, checking it against Monte Carlo estimates of the true rotated IoU. The third stage runs the comparisons below, including a shift setting: train on HRSC2016 ships and test on the DOTA ship class, where post-hoc calibrators fitted in-domain are known to be fragile.

Baselines

The naive reference is an unmodified KFIoU detector with raw scores, followed by the same detector with global temperature scaling fitted on a held-out split. This shows how much of the problem a single scalar fixes.

The strongest existing competitor is the multivariate logistic and beta calibration of Küppers et al. (Multivariate Confidence Calibration for Object Detection). We extend it fairly to oriented boxes by giving it the features $(z, c_x, c_y, w, h, \sin 2\theta, \cos 2\theta, \log(w/h))$. With those features it has explicit access to shape, so it is the most direct test of whether our analytic structure adds anything beyond a learned feature map. It is fitted on the same held-out split as all other post-hoc methods, and its regularization is tuned with the same search budget we give our link $g_\phi$. It must also be refit for each threshold $t$, because its formulation is tied to a single threshold. That refitting is itself part of the comparison.

The second strong competitor is MCCL (Multiclass Confidence and Localization Calibration for Object Detection), ported to rotated boxes. Its localization term uses rotated IoU, and its dropout samples are averaged in Gaussian space to avoid angle wraparound. It gets the same epochs and backbone, and we report its wall-clock training cost alongside ours.

The third competitor extends Bounding Box Regression With Uncertainty for Accurate Object Detection to five independent variances over $(x, y, w, h, \theta)$, with the same threshold-conditioned confidence link on top.

The key ablation keeps our head unchanged but replaces the closed-form expected overlap with a small MLP. The MLP maps the predicted uncertainties and box parameters to $q$. This isolates whether the Gaussian algebra matters or whether any learned uncertainty-to-confidence map would do. A second ablation drops the calibration loss and trains with the likelihood loss only.

No reported D-ECE result on any oriented benchmark was found on Lacuna. So the calibration bar is set by our own reproduction of the baselines. For context, the strongest reported calibration figure is MCCL reducing D-ECE on PASCAL VOC with FCOS from 11.88% to 6.02%, on horizontal boxes. For accuracy, the file reports $AP_{75}$ of 72.39% for RetinaNet with KLD on HRSC2016 and 77.35% mAP for KFIoU on DOTA-v1.0.

Beating the baselines means two things. First, our method should lower threshold-averaged D-ECE, and D-ECE in the highest aspect-ratio bins, below the extended multivariate calibrator, both in-domain and under the HRSC→DOTA shift. Second, $AP_{75}$ should stay within about half a point of the KFIoU or GauCho backbone.

Why not just…?

Why not just add aspect ratio and angle features to post-hoc multivariate calibration? This is the real threat, and we treat it as such. In-domain, with a large validation set, it may match us. Our argument for why it should fall short in some regimes has three parts. High-aspect-ratio objects are sparse, so a data-driven calibrator has few samples exactly where errors are worst; Küppers et al. already note data-efficiency concerns as feature dimension grows. A post-hoc calibrator sees only the box, not the per-instance uncertainty $U$, so it cannot separate a confidently localized ship from an ambiguous one of the same shape. And it must be refit for each threshold. The experiments settle this with the in-domain versus shift comparison and a sweep that shrinks the validation set. If the extended calibrator ties us in all three, we will say so.

Why not just port MCCL with dropout to rotated boxes? It would likely calibrate at IoU 0.5. It costs several passes per batch, its authors identify single-pass calibration as open, and it calibrates at a single threshold. Our comparison reports D-ECE at 0.75 and 0.9 against training time, so the reader can judge the tradeoff directly.

Why not just predict independent variances per parameter, as in He et al.? An independent angle variance cannot express that angle error only matters when width and height differ. It also inherits the wraparound of $\theta$. The independent-variance baseline, run with an identical confidence link, measures exactly this. If the gap closes, the coupling argument is wrong for calibration even though it holds for regression.

Decisive Checks

Three results would confirm the idea. The first is calibration error that rises with aspect ratio in stage one. The second is our method flattening that curve better than the shape-featured post-hoc calibrator under the HRSC→DOTA shift. The third is the closed-form head beating its MLP ablation, which shows that the Gaussian algebra, not just extra capacity, is doing the work. If any one of these fails, the proposal reduces to an audit of oriented detector calibration. That audit is still unreported in the literature.