Contractive Box Refinement via Closed-Form Worst-Case Losses

Modern object detectors rarely predict a bounding box in one shot. They start from a rough guess and nudge it several times: Cascade R-CNN chains three regression heads, and transformer detectors such as Deformable DETR and DAB-DETR let every decoder layer add a correction to the box handed over by the layer before. Each step is trained to be right on average for the boxes it typically receives. Nothing trains a step to behave well when the box it receives is somewhat wrong, for instance because an earlier layer drifted toward a neighbouring person in a crowd, or because fog corrupted the features. Our claim is that this missing property is cheap to train for and cheap to check. Ask each refinement step to minimize its worst-case L1 error over a small, scale-relative box of possible input errors. For a locally linearized step, the inner maximization has an exact closed-form solution. The result is a per-layer penalty on how strongly the step's output follows errors in its input. That penalty pushes every step toward being a contraction, meaning it shrinks incoming error instead of passing it on.

The payoff is twofold. During training, the penalty replaces the hand-tuned "quality ladder" of Cascade R-CNN with a single quantity: the radius of the uncertainty box at each layer. At test time, the same closed form yields a per-query bound on how much error survives each layer. That bound turns iterative refinement from a fixed number of steps into one with a stability certificate and a principled stopping rule.

Core Claim

Write the refinement performed by decoder layer or cascade stage $l$ as a map $F_l(b) = b + \Delta_l(b)$. Here $b \in \mathbb{R}^4$ is the incoming box in center–size coordinates, and $\Delta_l$ is the predicted correction, which depends on $b$ through the features pooled or sampled at $b$. Let $y$ be the matched ground-truth box. Let $C(b) = \{u : |u_x| \le \rho_l w,\ |u_y| \le \rho_l h,\ |u_w| \le \rho_l w,\ |u_h| \le \rho_l h\}$ be the set of incoming errors we want to tolerate, where $w, h$ are the box's width and height and $\rho_l$ is a radius per layer. The robust training target for layer $l$ is $\sup_{u \in C(b)} \|F_l(b+u) - y\|_1$.

Linearize around the observed box. Let $r = F_l(b) - y$ be the nominal residual and $A = I + \partial \Delta_l / \partial b$ the $4\times 4$ Jacobian of the refinement map. Then the supremum is attained by choosing a sign pattern for the four output coordinates, and the remaining maximization over $u$ is the support function of a box, which is a weighted L1 norm. This gives the exact expression

$$ \mathcal{L}^{\text{rob}}_l \;=\; \max_{s \in \{-1,+1\}^4} \Big[\, s^\top r \;+\; \big\| \operatorname{diag}(\rho_l w, \rho_l h, \rho_l w, \rho_l h)\, A^\top s \big\|_1 \Big]. $$

The maximum runs over only 16 sign vectors. $A$ costs four forward-mode Jacobian–vector products per layer, which is cheap next to the decoder itself.

The first term is the ordinary L1 loss. The second term is zero exactly when $A = 0$, that is, when the layer's output does not depend on errors in its input. That is the signature of a step that "snaps" to the object regardless of where it started. A trained layer with small $\|\operatorname{diag}(\rho)\,A^\top\|$ therefore has a certified local contraction factor $\kappa_l$. Over the set $C$, the worst-case L1 error after the layer is at most $\|r\|_1 + \kappa_l \cdot \text{(incoming error)}$. Composing these bounds across layers gives a per-query bound on final localization error, which we can compute on test images.

What is new is the specific object we make robust. The uncertainty is placed on the intermediate box passed between refinement steps, not on pixels and not on labels. The set geometry, a box scaled by object size, is chosen so that the worst case has an exact dual. The resulting quantity serves both as a training loss and as a test-time stability certificate.

Relation to Prior Work

The refinement side supplies the setting and the motivating evidence. Cascade R-CNN chains heads trained at IoU thresholds 0.5, 0.6 and 0.7, and shows that each stage sees a shifted, tighter distribution of offsets. It handles the shift by resampling positives, not by controlling how a stage reacts to bad inputs. Our per-layer radius $\rho_l$ plays the role of its IoU threshold, but as a set of tolerated errors rather than a filter on training samples. Deformable DETR introduced layer-wise box refinement in transformer decoders, and DAB-DETR made the box itself the query, with residual updates $\text{Box}_l = \text{Box}_{l-1} + \Delta\text{Box}_l$ and attention shaped by $w,h$. Because sampling locations depend on the box, $\partial\Delta_l/\partial b$ is well defined and includes the feature re-pooling effect that FA-RPN identified as important. We reuse these architectures unchanged and change only the loss. D-FINE represents each edge as a distribution refined across layers and distills deeper layers into shallower ones. It models uncertainty in the output, while we constrain sensitivity to the input. We treat it as the strongest host model and test whether the two combine.

The robust-optimization side supplies the method. Specifying and Solving Robust Empirical Risk Minimization Problems Using CVXPY shows that support functions of convex uncertainty sets turn worst-case risk into a single-level problem. Its geometric location sets, including a 1 km square, cut test-error inflation from over 6% to under 1% on Airbnb pricing. We borrow the support-function step, but apply it to a learned, iterated, nonconvex map, where only a local linearization is exact. Distributionally Robust Logistic Regression and Minimax Statistical Learning with Wasserstein distances show that worst-case objectives reduce to regularizers whose strength scales with the uncertainty radius. That is the same structure our penalty has.

Two pieces of prior work constrain the design. Lee and Raginsky note that exact duality fails for deep networks. This is why we restrict robustness to a 4-dimensional input, where linearization is checkable and the worst case can be verified by direct search. An Online Method for A Class of Distributionally Robust Optimization with Non-convex Objectives gets tractability for deep classifiers by reweighting samples with a KL penalty. That method reweights which proposals matter and does nothing about how a step responds to a perturbed box, so it addresses a different axis. We include it as a complementary baseline rather than our mechanism.

Research Program

The first component is the training loss. Each layer or stage's L1 term is replaced by $\mathcal{L}^{\text{rob}}_l$, while the GIoU and classification terms are kept. The Jacobian $A$ is computed with forward-mode differentiation through the deformable sampling or RoI pooling. Many implementations stop gradients on the box between layers. We keep that stop for the main training path but compute $A$ locally within each layer, so the penalty shapes each layer's own response. Radii $\rho_l$ shrink with depth. We will test both a fixed schedule derived from Cascade's IoU ladder, converting an IoU threshold into the corresponding scale-relative shift, and a schedule fitted to the empirical spread of errors arriving at each layer.

The second component is validating the linearization. For a held-out set of queries, we compare the closed-form worst case against a direct search: evaluating the true layer at all 16 vertices of $C$ and running a few steps of projected gradient ascent. We report the gap as a function of $\rho_l$. If the gap is large at useful radii, the method's claim to be "exact" collapses to "a heuristic upper bound." That would be a reportable negative result.

The third component is the certificate and the stopping rule. At inference we compute $\kappa_l$ per query and propagate the bound across layers. We stop refining a query once its certified remaining change falls below a pixel tolerance. We then measure how well the certificate is calibrated: how often does the realized error under injected box perturbations exceed the bound?

Baselines

The naive reference point is the unmodified host detector: Deformable DETR with iterative refinement, DAB-DETR, and D-FINE, trained with their published recipes. For Deformable DETR, the file reports 43.8 AP on COCO val. For DAB-DETR with a ResNet-50-DC5 backbone, it reports 45.7 AP. Each host is retrained in our codebase so that differences come only from the loss.

The strongest existing method is D-FINE, which reports 54.0 AP for D-FINE-L on COCO, and which already refines localization uncertainty layer by layer. Beating it means a statistically reliable gain in AP75 and AP90 on COCO, together with lower final error under injected intermediate-box perturbations, at matched epochs and matched GPU-hours. The few percent of extra compute spent on Jacobian–vector products is charged to our method by shortening its schedule accordingly. On high-overlap localization, Cascade R-CNN supplies a reference: it lifted AP90 from 7.1 to 15.8 over its FPN baseline. Cascade R-CNN itself is a second host, run with its 0.5/0.6/0.7 ladder against our radius schedule on the same backbone.

The baseline most dangerous to the proposal is box-jitter training. Here each layer receives its input box perturbed by noise drawn uniformly from the same set $C(b)$. The jitter variant gets the same radius schedule and the same number of extra layer evaluations per query that our Jacobian computation costs, and both are tuned with the same budget of radius sweeps. A second adversarial variant replaces our closed form with an explicit worst vertex found by evaluating all 16 corners.

Two ablations isolate the key ingredient. One replaces the set-shaped penalty with an unweighted Frobenius norm of $A$, which tests whether the scale-relative box geometry matters. The other applies our loss only to the last layer, which tests whether contraction must hold at every step. For sample reweighting, RECOVER-style KL reweighting of proposals is run on top of the host, alone and combined with our loss.

Why not just…?

Why not just jitter the input boxes during training? This is the real threat. Random jitter averages over the set, while our loss targets the worst point in it. In four dimensions, the worst corner may be rarely sampled, but a model might still learn enough from averages. Jitter also produces no test-time certificate. The matched jitter and explicit-vertex baselines settle this directly. If jitter matches our accuracy and perturbation robustness, the training contribution reduces to "jitter works," and only the certificate and stopping rule remain as contributions. We would report it that way.

Why not just add more refinement layers or a longer IoU ladder? Cascade R-CNN reports diminishing returns beyond three stages, and DAB-DETR notes persistent failures in very dense scenes. Both observations are consistent with later steps propagating early errors rather than correcting them. Our layer-wise error-trajectory plots test this directly: if extra layers already contract errors, measured $\kappa_l$ will already be small for the baseline, and our penalty will have nothing to do.

Why not just regularize the head or use a heuristic crowd loss? Weight decay penalizes parameter size, not sensitivity to the input box. Distributionally Robust Logistic Regression shows plain L2 to be a degenerate case of a worst-case objective, one that ignores the geometry of the uncertainty. Repulsion-style penalties target overlap with neighbours. The CrowdHuman benchmark still shows roughly 50% miss rate for FPN, which suggests such penalties leave much on the table. The Frobenius ablation tests whether our structured set beats a generic sensitivity penalty.

Why not just augment images with corruptions or stylization? Benchmarking Robustness in Object Detection shows that stylization raises retained performance, for example from 60.4% to 69.9% rPC on PASCAL-C. It acts on features rather than on the refinement dynamics, so the two are complementary. We run stylization alone and stacked with our loss, to see whether they add.

Proposed Experiments

The primary benchmark is COCO 2017. We report AP, AP75, AP90 and per-layer L1 error for all hosts. The decisive mechanistic check is a perturbation test: at inference we inject scale-relative offsets of controlled size into the boxes after layer two and measure the remaining error at the final layer. The proposal predicts a flatter curve than every baseline, including jitter, at matched clean AP.

For distribution shift, we evaluate COCO-C mPC and rPC across the 15 corruptions. The file gives a 30–60% performance loss for standard detectors, but no reported result found on Lacuna for COCO-C with DETR-style detectors, so the host baselines are measured fresh. For occlusion, we evaluate CrowdHuman log-average miss rate. The FPN reference is a ~50% miss rate, and there is no reported result found on Lacuna for D-FINE's CrowdHuman number. Crowds are where drift toward neighbours should make contraction matter most.

We also check the certificate's calibration: the fraction of perturbed queries whose realized error exceeds the composed bound should stay below a chosen level. We measure the AP–compute trade-off of certificate-based early stopping against fixed depth. Finally, we report the linearization gap between the closed form and direct vertex search across radii. The idea succeeds if four things hold together: clean AP is preserved or improved, the perturbation curve beats matched jitter, the bound is calibrated, and early stopping saves decoder compute without losing AP. It fails if jitter matches it on the first two.