Router Signals for Deciding Which Inspection Images a Human Must See

Automated bridge and pavement inspection does not remove the human inspector. It changes the inspector's job from walking the structure to reviewing images the model is unsure about. The real payoff of a defect detector is therefore how few images a person must look at while almost no cracks, spalls or rust patches are missed. That number depends on whether the model's confidence can be trusted on a structure it has never seen, with different concrete, weathering and lighting. Current bridge detectors increasingly use dynamic convolutions. These are a lightweight form of mixture of experts: a small network looks at each image and blends several candidate filter banks ("experts") into the one that gets applied. Separate work shows that this kind of soft blending is exactly what destroys confidence calibration under distribution shift.

We propose to test whether this blending can be turned from a liability into the escalation signal itself. The blending weights (the "routing coefficients") are a compact record of which filter regime the network chose for an image. When a new structure pushes those coefficients into combinations never seen in training, the image is flagged for human review, whatever the detector's confidence says. The research question is sharp and falsifiable. On held-out bridges, does routing novelty cut human review workload at a fixed defect recall, compared with confidence thresholds and with standard feature-space novelty scores computed on ordinary detectors?

Core Claim

The motivating tension comes from two groups of papers. Bridge detectors such as BD-YOLOv8s replace static convolutions with omni-dimensional dynamic convolution (ODConv), an extension of CondConv's input-dependent kernel mixing. BD-YOLOv8s reports 86.2% [email protected] on a single self-built dataset, but still flags vegetation as spalling and stains as corrosion, and never tests on unseen structures. Toward Calibrated Mixture-of-Experts Under Distribution Shift shows that soft-routed mixtures suffer "configuration collapse": many expert mixtures produce the same output, so a shift in which mixtures occur breaks calibration, while hard routing does not. That paper also shows that calibrating experts one at a time after training does not repair the mixture. It worked only on small classification models, and explicitly leaves open whether dynamic convolutions in dense detection behave the same way.

Our claim has two parts. The first is diagnostic: in dense defect detection, soft dynamic convolutions become worse calibrated than static convolutions when the model moves to a new bridge. We will measure this directly rather than assume it. The second is constructive. Let $r(x)$ be the concatenation of routing coefficients across all dynamic layers for image $x$, a vector of a few hundred numbers that the detector computes anyway. Fit a density $\hat p$ to $r$ on training structures and score each test image by

$$s(x) = -\log \hat p\big(r(x)\big).$$

An image goes to the human queue if $s(x) > \tau$ or if any detection falls in an uncertain confidence band. The two thresholds are set on source structures only. We predict that $s$ picks out the images whose detections will be wrong on a new structure better than detector confidence does, and at least as well as novelty scores on full feature maps, at almost no extra cost.

We deliberately do not claim that the mixture architecture improves accuracy under shift. OOD-CV-v2 finds no meaningful robustness difference between convolutional and transformer backbones under weather and context nuisances. Sparse Mixture-of-Experts are Domain Generalizable Learners argues the opposite for sparse vision-transformer mixtures on whole-image classification. We will report accuracy, but the proposal does not depend on who is right. Its value lies in the triage signal.

Research Program

The base detector is a YOLOv8s-scale model in three variants that share data, training schedule and parameter budget as closely as possible. The first uses static convolutions. The second uses ODConv soft routing in the positions BD-YOLOv8s uses. The third is a hard-routed version of the second, where the blend is replaced by the top-1 kernel with a straight-through gradient. The hard variant tests Wong et al.'s prediction that hard routing protects calibration.

The routing density $\hat p$ is a Gaussian mixture fit on routing vectors from training images. We will also try a per-layer version, because early layers should react to lighting and texture while later layers should react to defect type. Comparing which layers carry the novelty signal is itself an analysis output: it tells us whether shift enters through low-level appearance or through semantics.

The operational metric is review workload. This is the fraction of test images a human must view so that at least 95% of images containing a defect are either correctly detected or escalated. It is computed on structures never seen in training, with thresholds frozen from source-structure validation. We also report detection calibration error computed on IoU-matched confidences, following the IoU-aligned view of detector confidence in MoCaE, plus [email protected] and [email protected]:0.95.

The primary benchmark is CODEBRIM from the meta-learning concrete defect paper. It has 1,590 images from 30 bridges, with boxes for five overlapping defect types, so leave-bridges-out splits give natural cross-structure shift. On top of held-out bridges we apply compounded nuisances (fog, low light, glare, combined), because OOD-CV-v2 reports that combined nuisances hurt more than the sum of their individual effects. A secondary pixel-level setting ports the same three convolution variants into a crack segmentation network. It trains on Crack500 and tests on CFD and DeepCrack, the cross-dataset protocol used by UP-CrackNet and CAC. This setting checks whether routing novelty flags images with shadows, oil stains and unfamiliar textures, which the file reports as the dominant crack false-alarm sources.

Baselines

The naive reference is the static-convolution detector with a single confidence threshold chosen on source bridges. Images with any detection above the threshold, or any detection in the uncertain band, go to the human. This is what a deployment would do by default.

The strongest architectural competitor is the ODConv soft-routed detector in the style of BD-YOLOv8s, triaged by confidence alone. On its own in-domain dataset it reports 86.2% [email protected] and 56% [email protected]:0.95 at 84 FPS. We cannot reproduce those numbers because that dataset appears to be self-built. On leave-bridges-out CODEBRIM there is no reported result found on Lacuna for detection mAP, calibration or review workload. The only CODEBRIM number in the file is roughly 72% multi-target classification accuracy for MetaQNN-2, which we will reproduce as a sanity check on our splits.

The strongest triage competitor is a feature-space novelty score on the static detector. This is a Mahalanobis-style Gaussian density on pooled backbone features at the same depths where the dynamic layers sit, combined with the same confidence band. A calibration competitor is Wong et al.'s entropy-balanced adversarial reweighting applied to the soft-routed detector; on CIFAR-10H it lowers ECE from 0.281 to as low as 0.065 in combination with other robust methods. A post-hoc competitor is isotonic regression of detection confidence against IoU on source validation, following MoCaE. A shift-removal competitor is instance-specific normalization calibration from InstCal, which raises GTA5→Cityscapes mIoU from 35.7% to 42.2% in one forward pass. A three-member deep ensemble is included as a cost-unconstrained upper reference.

All methods share the same splits, augmentation and number of tuning trials, and all thresholds are chosen on source bridges only. The ablation that isolates our key ingredient fits the same density on the router's input rather than its output. The router's input is the globally pooled feature vector that the routing network projects into coefficients. If $s$ on coefficients beats $s$ on the router input, the learned routing projection carries information that raw pooled features do not.

Beating the field means one thing: lower review workload at 95% recall on held-out bridges than both confidence-only ODConv and feature Mahalanobis on the static model, with non-overlapping confidence intervals across bridge folds, and without adding more than a few percent latency.

Why not just…?

Why not just use hard routing, which Wong et al. already showed stays calibrated? This is a serious threat and we test it head on as the third variant. If hard routing alone brings calibration back to static-model levels and confidence thresholding then matches our workload, the constructive half of the proposal is unnecessary. The diagnostic result, the first evidence on configuration collapse in dense detection, would stand on its own. We expect hard routing to fix the confidence scale but not to reveal which images come from an unfamiliar environment. A well-calibrated model can still be confidently wrong on a stain it has never seen, which is exactly the failure BD-YOLOv8s reports.

Why not just use a standard novelty detector on ordinary features? This is the most likely way the idea fails. Mahalanobis distance on backbone features is cheap and well understood. Routing vectors are a much smaller summary that the network already computes, so a tie would still favor them on edge hardware, but a clear loss for routing kills the MoE angle. The router-input ablation distinguishes the two cases: "routing is just a compressed feature" versus "routing encodes regime choice."

Why not adapt the detector to the new bridge instead of escalating images? Single-pass normalization calibration such as InstCal avoids the latency of gradient-based test-time optimization. MonoTTA, however, reports that adaptation struggles when positives are scarce, and defect pixels are often below 0.1% of a frame. Adaptation and escalation also answer different questions: adaptation may raise mAP while leaving the inspector unable to know which images to trust. We run InstCal-style calibration both as a competitor and combined with routing novelty, and report whether it reduces workload on its own.

Why not a calibrated ensemble? MoCaE shows that calibrated heterogeneous ensembles beat naive ones on COCO. They multiply memory and latency, though, and their isotonic calibration is fit on in-domain data, which is precisely what shifts. The three-member ensemble gives the ceiling that our single-model signal should approach.

Decisive Checks

Three results decide the proposal. First, on leave-bridges-out CODEBRIM, the soft-routed detector's IoU-aligned calibration error must rise more than the static detector's under shift. If it does not, configuration collapse does not transfer to dynamic convolutions, and the diagnostic motivation is wrong. Second, routing novelty must separate images whose detections are wrong from those that are right, measured by AUROC on held-out bridges and on compounded nuisances, better than detector confidence and at least as well as feature Mahalanobis. Third, and most important, review workload at 95% recall must fall relative to every baseline across bridge folds, and the router-output score must beat the router-input score. The crack cross-dataset setting, Crack500 to CFD and DeepCrack, then checks that the signal flags shadow- and stain-heavy images rather than acting as a generic image-statistics detector. If the second and third checks fail while the first holds, we publish the negative result: soft dynamic convolutions harm trust under shift, and deployments should switch to hard routing rather than try to exploit the blend.