Local Spectral Contrast for Tiny Object Detection Under Domain Shift

When an object in an aerial or surveillance image covers only a handful of pixels, a detector cannot see its shape. What remains is a sharp local spike in brightness, and in frequency terms that spike spreads energy across the fine-detail (high-frequency) bands. Recent tiny-object detectors exploit exactly this signal. The trouble starts when the camera moves to fog, night or a different sensor. Those conditions rescale and pollute the high-frequency bands, and the leading frequency-based methods for cross-domain robustness treat high frequencies as disposable "style." The result is that the only signal a tiny object has is the one domain generalization tries hardest to remove or randomize.

This note proposes that a tiny object's cross-domain signature is not its absolute high-frequency energy. It is the ratio of that energy to the energy of its immediate background in the same frequency band. Fog, low light and gain changes multiply band energies over regions much larger than the object, so the ratio largely survives. We propose to build this ratio directly into the detector's feature pyramid, the stack of feature maps at several resolutions that detectors use to find objects of different sizes. We pair it with a training-time augmentation that randomizes band energies with spatially smooth fields, so the ratio is preserved without pasting clean patches. The claim is testable: gains over existing frequency-based domain generalization should appear mainly on objects of 2 to 16 pixels, and mainly under unseen shifts. They should shrink on in-domain data and on larger objects.

Core Claim

The literature disagrees about where the transferable frequencies are. FSDR randomizes the very low and very high DCT bands for synthetic-to-real segmentation. It argues that domain-invariant geometry lives in the middle bands. High-Frequency Component Helps Explain the Generalization of CNNs finds that high-frequency cues generalize poorly, yet removing them drops COCO detection mAP from 35.6% to 27.5%. Generalized UAV Object Detection via Frequency Domain Disentanglement finds middle and high bands more invariant than low bands under illumination shift. SET shows that high-frequency background clutter is the main distractor for tiny objects.

Our resolution is that these papers are measuring different things. Absolute band energy is domain-variant at every band. The local contrast of a small region against its surroundings, computed within a band, is closer to invariant when the shift acts smoothly in space. For band $b$ of a multi-scale decomposition, location $x$, and local band energy $E_b(x)$, define

$$ c_b(x) = \frac{E_b(x)}{\operatorname{med}_{y \in N(x)} E_b(y) + \epsilon}, $$

where $N(x)$ is an annulus of neighboring positions around $x$ excluding the center, $\operatorname{med}$ is the median, and $\epsilon$ is a small constant. If a domain shift multiplies band $b$ by a gain $g_b(x)$ that varies slowly compared with the annulus radius, the gain cancels in $c_b$. Additive sensor noise inflates the denominator and lowers contrast, but it preserves the ranking of target over background. A detector that attends to $c_b$ instead of $E_b$ should therefore transfer better for exactly those objects whose only evidence is a local spike.

Research Program

The first component is a Local Band Contrast module inserted into the lateral connections of the feature pyramid. It sits in the same position as the High Frequency Perception module of HS-FPN, which high-pass filters features with a fixed cutoff $\alpha=0.25$ and uses the absolute response as channel and spatial attention. We keep the channel-and-spatial attention structure, which serves as our method source. The absolute response is replaced by $c_b$ computed over three or four learnable-width bands of a Laplacian pyramid on the feature map. This replaces one hand-tuned cutoff with a small set of bands whose contrast is normalized. It speaks to the cutoff-sensitivity problem noted for HS-FPN. A robust median rather than a mean in the denominator is chosen so that an isolated target does not raise its own reference level.

The second component is Smooth-Field Spectral Randomization, a training augmentation used only on source images. For each band we sample a random log-gain field that is smooth at a scale several times larger than the largest tiny object. We multiply the band by it and then add band-limited noise at random strength. Because the field is smooth, a target and its surroundings receive nearly the same gain, so local contrast is preserved without masks or copy-back. This contrasts with FSDR, which histogram-matches whole DCT bands to reference images. That nonlinear remapping of coefficient tails is exactly where point-like impulses sit, and we will measure whether it compresses them. It also contrasts with OA-DG, whose OA-Mix protects low-saliency objects by keeping their clean pixels while the rest of the image is mixed. That spatial approach creates a clean-versus-stylized boundary the detector could learn as a shortcut.

The third component is diagnostic. Given ground-truth centers of tiny objects in clean and corrupted versions of the same images, we measure how much $E_b$ and $c_b$ change at those centers and in background. We also measure the internal attention maps of HS-FPN, SET and our module. This tests the mechanism directly rather than only through AP.

Label assignment is held fixed across all methods, using the normalized Wasserstein distance with ranking-based assignment from NWD-RKA. It reached 24.7 AP on AI-TOD-v2 and avoids the zero-positive-anchor collapse that IoU causes for tiny boxes. Otherwise, domain-induced localization jitter would confound the comparison.

We expect a known failure mode. CodeMamba reports that targets beside bright, sharp background edges get absorbed into its background model. A median over an annulus that straddles such an edge will inflate the denominator in the same way. We will report this case separately rather than hide it. If it dominates the errors, we will test a directional annulus variant that uses the minimum over a few sectors.

Baselines

The naive reference is a Cascade R-CNN with ResNet-50, a standard FPN and NWD-RKA assignment, trained on the source domain with standard flips and scaling. On AI-TOD this detector family with a plain FPN is reported at 20.2 AP by HS-FPN. It tells us how much any cross-domain method recovers relative to doing nothing.

The strongest in-domain tiny-object competitors are HS-FPN and SET. HS-FPN reports 23.6 AP on AI-TOD with Cascade R-CNN ResNet-50. SET reports a 3.2 AP gain over RFLA and 26.6 AP with DINO-5scale. Neither has been evaluated under domain shift, and no reported result was found on Lacuna for either on any corrupted or cross-dataset tiny-object split. Both matter because they already exploit high frequency, so they test whether contrast normalization adds anything over absolute high-frequency attention. Both will be re-run in our Cascade R-CNN plus NWD-RKA pipeline with the same schedule. SET will use its ground-truth-mask background smoothing, which is legal on the labeled source.

The strongest cross-domain competitor is the frequency disentanglement method of Wang et al.. On UAVDT trained on daylight, it reports 29.4% overall AP against 20.4% for YOLOv5, and 20.7% night AP against 4.8%. We will reproduce it in two ways. First, we run their YOLOv5 protocol on UAVDT so the 29.4 and 20.7 numbers anchor our reproduction. Second, we port their learnable spectral filters and instance contrastive loss into our Cascade R-CNN pipeline. We also include FSDR's band randomization applied to detection and the DFF module from Deep Frequency Filtering, a learned global frequency mask on intermediate features validated only on classification and person re-identification. These represent the frequency approaches that do not separate target from background. OA-DG is the spatial object-aware competitor; it reports 21.8 mPC on Cityscapes-C and 31.8 on the Diverse Weather Dataset.

Every method gets the same backbone initialization, epochs, input resolution, and a matched budget of eight hyperparameter trials selected on a held-out source-domain split. No method sees target-domain data during tuning.

Two ablations isolate the key ingredient. The first keeps the pyramid module but feeds it absolute $E_b$ instead of $c_b$, with everything else identical. The second uses the augmentation with spatially constant gains, making it a global randomization like FSDR's. Beating the strongest competitor means exceeding the ported Wang et al. model on the very-tiny and tiny AP buckets under unseen shifts by more than seed variance across three seeds. On UAVDT's own protocol, it means exceeding the reported 20.7 night AP.

Why not just…?

Why not just apply FSDR or amplitude-spectrum mixing to a tiny-object detector? This is the most serious threat, because a global band gain also preserves local contrast. Our method could reduce to "FSDR plus a good tiny-object detector." The spatially-constant-gain ablation answers this directly. If it matches the smooth-field version and the $c_b$ module adds nothing on top, we concede that global spectral randomization suffices. The two places we expect a difference are spatially varying shifts such as patchy fog and vignetting, and FSDR's histogram matching of band tails. The diagnostic will measure the latter's effect on target impulses.

Why not just protect the objects spatially, as OA-Mix does? OA-DG shows that preserving clean pixels for weak objects helps on Cityscapes-C, moving the baseline from 15.4 to 21.8 mPC. Its saliency score, however, depends on visible contrast, which is unreliable at 2 to 8 pixels, and clean pasted regions carry a stylistic seam. We run OA-DG under the identical pipeline. We also run a "copy-back" variant that pastes clean tiny-object crops into FSDR-augmented images, and we check whether its false positives concentrate on unstyled texture patches.

Why not just normalize the input, for example with local contrast equalization? Spatial local normalization rescales all frequencies together. Fog, by contrast, attenuates fine detail more than coarse structure, and night adds noise mainly at fine scales, so a single gain cannot undo a band-specific shift. We include adaptive histogram equalization of inputs as a baseline. If it closes most of the gap, the band decomposition is unnecessary and we will say so.

Why not just train with synthetic fog and noise corruptions? That works when test corruptions resemble training ones, which is not domain generalization. Our AI-TOD corruption protocol holds out corruption families: models trained with weather corruptions are tested on sensor noise and blur, and vice versa. This lets us report the gap between seen and unseen corruption families.

Proposed Experiments

Three settings are used. The first is UAVDT following the daylight-to-night-and-fog protocol of Wang et al., reporting overall, night and fog AP broken down by object size. The second is AI-TOD, with average object size 12.8 pixels, trained clean and tested on a corrupted copy built the way Cityscapes-C was constructed, with corruption families held out as above. We report mean performance under corruption alongside the very-tiny and tiny AP buckets. The third is cross-dataset transfer from VisDrone2019 to AI-TOD on their shared vehicle and person categories, and the reverse. The reference points here are SR-TOD's 27.3 in-domain AP on VisDrone2019 from Visible and Clear and the AI-TOD numbers above. No reported cross-dataset result was found on Lacuna for this pair.

The decisive checks are these. First, the full method should beat the ported frequency-disentanglement model on the very-tiny bucket under unseen shifts, while roughly tying on medium objects. A uniform gain across sizes would suggest a generic regularization effect rather than our mechanism. Second, the $E_b$-versus-$c_b$ ablation must show the contrast input carrying most of the gain. Third, the diagnostic must show that $c_b$ at target centers changes substantially less between clean and corrupted images than the attention signals of HS-FPN and SET. Finally, performance on targets adjacent to strong edges is reported as its own subset, so the known weakness of median-based contrast is measured rather than averaged away.