Object detectors trained on clear daytime street scenes lose most of their accuracy on small, distant objects when deployed in fog, rain, night, or a new city. One mechanical reason is that the same corruption hits objects of different sizes unequally. A blur of two pixels barely changes a 200-pixel car but erases a 12-pixel pedestrian. Augmentation methods that try to teach robustness by corrupting whole training images therefore teach very little about small objects, which are exactly where models fail. We propose a "second look" detector. A standard detector first proposes candidate boxes. Each small or uncertain candidate is then cut from the full-resolution image, resampled to one fixed canonical size, and re-scored by a dedicated head. Training corruptions for that head are applied after resampling, so their strength is relative to the object rather than to the image. The pixels surrounding the object inside the crop are randomly replaced with background from other images, so the head can use context without depending on any one domain's background.
In two sentences: rescale every hard candidate to the same size before judging it, and practice on corrupted, background-swapped versions of that rescaled crop. That way, robustness learned at one object size carries over to all sizes, and context helps recognition without becoming a shortcut. The test is whether this beats the strongest object-aware generalization method on corrupted and weather-shifted driving benchmarks. It must also beat simpler explanations of the same gain: upscaling the input, tiling, or the object-aware augmentation alone.
Core Claim
Two ideas meet here. The first is region-of-interest warping, the family of methods that locate a region coarsely and resample it to a canonical scale before fine analysis. The second is domain generalization, meaning training on source domains and deploying to unseen targets with no target data at all. Our claim is that warping is valuable under domain shift mainly because it makes corruption and context scale-consistent, not simply because it adds pixels. A head that only ever sees objects at one size can be trained against a single, well-calibrated corruption distribution. A whole-image detector instead sees a different effective corruption at every object size.
The prediction that would falsify this: if canonical-frame augmentation adds nothing over warping with ordinary image-level augmentation, the idea reduces to resolution recovery and is not interesting.
Prior Work and the Gap
Warping methods motivate the mechanism. Each shows that scale normalization strongly helps small-object discrimination, but only within a single domain:
- Clustered Object Detection in Aerial Images (ClusDet) proposes dense clusters, predicts a per-cluster scale factor, and re-detects on rescaled chips. It reaches 28.4 AP on VisDrone against 21.4 for Faster R-CNN and 24.4 for uniform tiling, while admitting that it misses isolated objects.
- AutoScale: Learning to Scale for Crowd Counting rescales the single densest region with an unsupervised objective.
- Consistent Scale Normalization for Object Recognition restricts training and inference to a fixed object-size range at each pyramid level, lifting COCO Faster R-CNN from 38.0 to 42.9 mAP.
- Two-stage text detectors, Cascaded Segmentation-Detection Networks for Word-Level Text Spotting and All you need is a second look, use the same crop-and-resize pattern.
We reuse this pattern but apply it per instance rather than per cluster. That sidesteps ClusDet's isolated-object failure, which matters because target density changes under domain shift.
Domain generalization methods for detection constrain the design:
- Object-Aware Domain Generalization for Object Detection (OA-DG) shows that whole-image augmentations destroy small, low-contrast objects. Its remedy is to shield low-saliency objects from mixing and to contrast background patches against each other. It leaves those objects at their native degraded scale.
- Towards Domain Generalization in Object Detection (RAPT) reweights proposals so that region features decorrelate from background features. It operates on pooled features from fixed-resolution inputs.
Both treat background as the enemy. OD3: Optimization-free Dataset Distillation for Object Detection points the other way: its scale-aware context extension shows that small objects need proportionally more surrounding context to be identifiable at all. We address this contradiction directly. We keep a context ring around each warped object and swap its contents across images, so context stays available as a signal while its domain-specific statistics stop being predictive.
FAMNet argues that cross-domain invariance in medical images lives in mid-frequency bands. Bilinear resampling reshapes an object's spectrum relative to the network's fixed filters, and we analyze this rather than ignore it.
Research Program
The detector is a Faster R-CNN with a ResNet-50 backbone and feature pyramid, chosen to match OA-DG and RAPT. The first stage runs at the usual training resolution.
Candidate selection and warping. Candidates enter the second look if their box is smaller than a size threshold or their first-stage confidence falls in an uncertain band. Each is cropped from the full-resolution source image with a margin of $m$ times the box size, then resampled to a canonical $S \times S$ patch. The canonical size $S$ is chosen so the object lands in the backbone's best-performing size range, which we measure on clean source validation data, following the size-range logic of instance scale normalization. The margin $m$ is sampled during training and selected on held-out source domains at test time.
Second-look head. A light head, sharing the backbone weights, re-scores and refines each patch. Its outputs are fused with first-stage scores before non-maximum suppression.
Training augmentations. Two augmentations are applied only inside the canonical frame:
- Object-relative corruptions. We use the blur, noise, contrast, fog, and low-light families from AugMix and Cityscapes-C style pipelines. Because every patch has the same object size, a given severity means the same thing for every object.
- Context swap. The ring between the object box and the crop border is replaced, with some probability, by a ring cut from another training image or another source domain. A consistency loss between the original and swapped views asks the head to give the same prediction.
Spectral analysis. For matched objects, we compare the radially averaged power spectrum of warped patches against native-scale features under clean and corrupted conditions. The question is whether the second look's robustness tracks preservation of mid-band energy, as FAMNet's account would predict. This is diagnostic, not a design dependency.
Baselines
Naive reference. A Faster R-CNN with the same backbone, trained on source data with standard flips and resizing. Every number is reported relative to it.
Strongest existing method: OA-DG. It is the strongest competitor because it targets the same failure of small, fragile objects under shift. OA-DG reports 21.8 mPC (mean performance under corruption) on Cityscapes-C, against 15.4 for its baseline and 21.0 for FSCE, and 31.8 mPC on the Diverse Weather Dataset, against 26.2 for its baseline and 28.7 for CDSD. We will run its released configuration with our backbone and schedule, and quote both its reported and reproduced numbers. Beating it means exceeding the reproduced mPC on both benchmarks, with the gain concentrated in the small-object bucket. A gain of at least 1.5 mPC overall would be meaningful against the 0.8 margin OA-DG held over FSCE.
Classic domain generalization: RAPT. RAPT reports 0.414 mAP training on BDD100k, Cityscapes and KITTI and testing on Sim10k. We run that protocol and quote that figure.
Resolution controls. Every pipeline gets the same full-resolution access and an equal test-time compute budget measured in FLOPs. Any variant using extra second-look compute is compared against a baseline given the same FLOPs as an upscaled input or tiles. Tuning budgets, defined as the number of hyperparameter trials, are matched. All selection uses held-out source domains, never targets.
Ablations isolating the key ingredient. Four variants separate the contributions:
- warping with standard image-level augmentation only, which isolates resolution recovery;
- canonical-frame augmentation without the context swap;
- context swap at margin zero, meaning no context;
- the full model applied to all candidates versus only small or uncertain ones.
Why not just…?
Why not just upscale the whole image at test time, or train with multi-scale jitter? This is the most serious threat, because part of any warping gain is recovered resolution. The FLOP-matched upscaled baseline settles it. If the warp-only ablation already matches the full model, and upscaling matches the warp-only ablation, the canonical-frame claim fails and we will say so. What upscaling cannot do is make corruption severity object-relative during training. Its gap to the full model, measured per object-size bucket, is the evidence for or against our mechanism.
Why not just tile the image, as uniform-grid detectors do? ClusDet's own comparison shows tiling trailing scale-adaptive chips in-domain, 24.4 versus 28.4 AP on VisDrone. Tiling also leaves objects at whatever scale the tile gives them, so it does not produce a canonical frame. We include FLOP-matched tiling as a baseline. We expect it to help small-object AP somewhat but not to close the corruption gap.
Why not just use OA-DG's object-aware mixing, which already protects small objects? OA-Mix protects fragile objects by perturbing them less, which means the model never practices recognizing them under strong corruption. Our approach corrupts them at a calibrated strength in their own frame. The methods are complementary, so we also report OA-DG combined with our second look. If the combination gains nothing over OA-DG alone, the second look is redundant with saliency gating.
Proposed Experiments
Primary benchmarks.
- Cityscapes to Cityscapes-C, reporting mPC overall and per object-size bucket.
- The Diverse Weather Dataset, trained on clear daytime and tested on night, rain, fog and dusk.
- The classic multi-source protocol (BDD100k, Cityscapes, KITTI to Sim10k), reporting mAP against RAPT's 0.414.
Density and context shift. VisDrone-trained detectors are tested on UAVDT, which probes isolated-object behaviour that cluster methods miss. No reported result was found on Lacuna for VisDrone-to-UAVDT cross-dataset detection, so we establish it with ClusDet-style and tiling baselines. The scaffolding for tiny-object evaluation comes from PointTinyBenchmark.
Decisive checks. The proposal succeeds only if four conditions hold:
- The full model beats reproduced OA-DG on Cityscapes-C and DWD under matched compute.
- The gain over the warp-only ablation is positive and concentrated on small objects. This is the test that separates our mechanism from resolution recovery.
- Gains do not shrink as corruption severity rises. A head that has learned object-relative corruptions should degrade more gently at high severities.
- The context swap beats both zero-margin crops and unswapped context on the Context Shift split from RAPT's protocol. This would show that the context tension between RAPT and OD3 can be resolved by randomizing context rather than removing it.
The spectral analysis then tells us whether robustness follows mid-band preservation. Either answer is informative for future warping designs.
References
- Clustered Object Detection in Aerial Images
- AutoScale: Learning to Scale for Crowd Counting
- Consistent Scale Normalization for Object Recognition
- Cascaded Segmentation-Detection Networks for Word-Level Text Spotting
- All You Need is a Second Look: Towards Tighter Arbitrary Shape Text Detection
- Object-Aware Domain Generalization for Object Detection
- Towards Domain Generalization in Object Detection
- OD³: Optimization-free Dataset Distillation for Object Detection
- FAMNet: Frequency-aware Matching Network for Cross-domain Few-shot Medical Image Segmentation