Receptive-Footprint Weighting for Cross-Domain Oriented Detection

Detectors for aerial and satellite imagery break when moved to a new sensor, city, or season. Ships, planes and vehicles keep their shape across these shifts, but the ground, sea and rooftops around them change completely. The usual fix in domain adaptation is to train the detector so that a small classifier cannot tell which domain a feature came from. When that matching is done over the whole image, it mostly matches the background. Prior work fixes this by aligning only "foreground" pixels. But in overhead imagery objects are rotated and slender, and the features we align are produced by multi-scale dilated filters, which sample pixels far from the location they describe. A feature sitting on a ship's hull at a large dilation rate is mostly a description of the water around it. A foreground mask, even a perfectly rotated one, does not stop that feature from carrying background into the alignment.

This note proposes a fix for that leak. We split the alignment-tap features into channel groups, one per dilation rate. Each feature at each rate then gets a weight equal to the fraction of its own sampling footprint that falls inside a rotated object box. Rate-1 channels on a small car are aligned almost fully. Rate-4 channels on the same car are aligned almost not at all. The weights come from geometry the detector already predicts, so the method adds no new labels. It is a direct test of a tension that the literature leaves open. Dilation helps with scale (PSConv). Wide receptive fields hurt small aerial objects (SADA-Net). And wide context feeds background into cross-domain alignment (Decoupled Adaptation).

Core Claim

Foreground-aware alignment should be decided per receptive field, not per pixel. The claim has two testable parts.

The first part is about where the leak happens. In a dilated multi-scale layer, the domain gap measured on high-rate channels inside rotated object boxes behaves like the background gap, not the object gap. A per-rate discriminator trained to tell domains apart should find high-rate channels inside the box almost as easy to separate as background, and much easier than low-rate channels inside the box.

The second part is about the fix. Weighting each rate's alignment loss by the share of its footprint that lies inside the rotated box improves target-domain oriented detection over a uniform rotated-mask weighting with the same detector, data and discriminators. The gain should be largest on small objects and on very elongated ones, where footprints spill out of the box the most.

If the first part fails, so that high-rate in-box features are not background-dominated, the method has no reason to work, and we will report that as the result.

Research Program

Base detector. We use Oriented R-CNN with a ResNet-50 backbone. Oriented R-CNN generates rotated proposals from horizontal anchors by predicting two midpoint offsets. That gives us rotated foreground regions cheaply, in both domains.

Choice of dilated layer. PSConv varies dilation cyclically across input channels and shifts the pattern across output channels. As a result, its output channels mix rates and cannot be cleanly assigned to one. For the alignment tap we therefore use a channel-partitioned dilated block in the style of Lou et al., where each channel partition has exactly one dilation rate. We insert it after each FPN level with rates {1, 2, 4, 8}. PSConv appears separately as a backbone variant, so we can tell whether in-layer multi-scale features help adaptation at all, independently of the weighting.

Rotated foreground mask. Let $M(q)\in[0,1]$ be a soft mask at input location $q$. In the source domain, $M$ is the ground-truth oriented boxes rasterized. In the target domain, $M$ is the rasterized top oriented proposals, each scaled by its classification confidence.

Footprint weight. For feature location $p$ in the partition with dilation rate $r$, let $S_r(p)$ be the set of input locations covered by that feature's dilated 3×3 sampling grid at the level's stride. The weight is

$$w_r(p) = \frac{1}{|S_r(p)|}\sum_{q\in S_r(p)} M(q),$$

that is, the fraction of the footprint inside a rotated object.

Alignment loss. Each partition $f_r$ has its own pixel-wise domain discriminator $D_r$, connected through a gradient-reversal layer (a layer that flips the gradient sign so the backbone learns to fool the discriminator). The alignment loss is

$$\mathcal{L}_{align}=\sum_r\sum_p w_r(p)\,\mathrm{BCE}\big(D_r(f_r(p)),\,d\big),$$

where $d$ is the domain label and BCE is binary cross-entropy. The weighting has one useful side effect for target-domain noise. Wide footprints are only aligned when a confident, large rotated box covers them, so a spurious low-confidence target box can only disturb low-rate channels over a small area.

Measurement before any training. We first train frozen-backbone per-rate discriminators on the unadapted detector and report their domain-classification accuracy, binned by rate, by in-box versus background, and by object size and aspect ratio. This tests the first part of the claim.

Open question. Gap 5 in the literature asks whether rotation-sensitive regression features should be aligned at all. Following the observation in Rotation-Sensitive Regression that classification and regression want different invariances, we run one secondary ablation that applies the loss only to features feeding the classification head.

Baselines

The naive reference is Oriented R-CNN (ResNet-50) trained on source only and evaluated on the target. It sets the size of the domain gap. For scale, the file reports supervised in-domain results of 75.87% mAP on DOTA and 96.50% mAP on HRSC2016 for this detector (Oriented R-CNN). A target-supervised model gives the upper bound.

Global adversarial alignment is the standard image-level discriminator with gradient reversal on all FPN features. It represents the approach that Subspace Alignment and Munir et al. describe as spatially blind.

The strongest existing method is center-aware alignment from Every Pixel Matters. It weights pixel-level adversarial alignment by predicted objectness and centerness. It reports 40.2% mAP on Cityscapes→Foggy Cityscapes (ResNet-101) against 25.6% without adaptation, and 49.0% on Sim10k→Cityscapes (VGG-16). We port it to the oriented detector in two forms: with its original axis-aligned maps, and with centerness computed inside rotated boxes. There is no reported result on Lacuna for any domain-adaptive oriented detector on DOTA, HRSC2016, UCAS-AOD or FAIR1M, so every cross-domain number here will be produced by our own runs.

We also run the input-space route of JCST-UDA. It translates images with a luminance-preserving CycleGAN and reports Faster R-CNN on Gaofen-2→DSOD rising from 0.81 to 0.85 mAP. We run it as published, on its own DSOD setting, and also feed its translated images into our oriented detector.

The key ablation is uniform rotated-mask alignment. It is identical to our method except that $w_r(p)=M(p)$ for every rate: same partitioned layer, same per-rate discriminators, same target masks. Beating this is the only result that supports the footprint idea. Anything else could be explained by "rotated masks help".

Fairness controls. All methods share the backbone, schedule, crop size (1024×1024 patches), augmentation, and a fixed tuning budget of eight runs over the alignment weight and learning rate. Following standard unsupervised-adaptation practice, that budget is selected on a small labeled target validation split, and this choice is reported. Results are averaged over three seeds.

What counts as winning. Our method beats the strongest baseline if it exceeds both uniform rotated-mask alignment and rotated center-aware alignment on target mAP by more than the seed standard deviation, on at least three of the four transfers below.

Why not just…?

Why not just restrict center-aware alignment to rotated boxes? This is the obvious combination and the most likely competitor, so it is in the baselines in both axis-aligned and rotated forms. It fixes the shape of the mask but not the leak. A rate-8 feature at the exact center of a 20-pixel vehicle still samples mostly ground, whatever mask surrounds it. If the rotated version matches our method, the pre-training measurement should already have shown that high-rate in-box features are not background-dominated, and we would drop the claim.

Why not just erode the rotated mask by a fixed margin? This is a real threat. A single erosion margin reproduces footprint weighting exactly for one dilation rate. We therefore treat erosion as a tuned baseline and sweep the margin across the footprint radii of all four rates, giving it the full tuning budget. Our method can only outperform it where different rates need different margins at the same location, which should show up on mixed-size scenes in DOTA and FAIR1M. If the best single margin matches us, the honest conclusion is that a simpler rule suffices, and we would report it.

Why not just translate images with a style-transfer network? JCST-UDA reports that plain CycleGAN degrades fine texture: SSIM 0.85 and PSNR 23.99, against 0.99 and 30.82 for their luminance-preserving variant. Their gains are 2–4 mAP across detectors. Input-space translation is complementary to feature alignment rather than a replacement, so we report it both alone and stacked under our method.

Why not just rely on FPN and skip dilated layers entirely? If the dilated tap does not help, the footprint question disappears, because standard 3×3 features have nearly uniform footprints. The FPN-only arm with uniform rotated alignment answers this directly. PSConv's reported COCO gain (37.3→39.4 AP for Mask R-CNN, PSConv) was measured in a single domain and does not show it survives a domain shift.

Proposed Experiments

The file contains no established benchmark for oriented detection across domains, so we construct transfers from the datasets it describes, keeping only shared categories. DOTA→HRSC2016 uses ship only; it tests elongated objects against harbor and sea clutter (DOTA). DOTA→UCAS-AOD uses plane and vehicle and is dominated by small, dense objects. DOTA→FAIR1M uses the coarse airplane, ship and vehicle classes, a larger sensor and resolution change (FAIR1M). Where annotation format permits, we add Gaofen-2→DSOD so we can compare directly against JCST-UDA. The protocol for DSOD's runway class is to be determined: runways are the elongated, background-heavy case, but the file does not say whether DSOD provides oriented annotations. As a check that the method does no harm on upright objects, we also run Cityscapes→Foggy Cityscapes with horizontal boxes, where our method should reach rather than fall below the 40.2% reported for center-aware alignment.

Metrics. We report target oriented-box mAP, plus mAP broken down by object size (with DOTA's 10–50 pixel small band reported separately) and by aspect ratio.

Decisive checks. Three results matter most:

  • the per-rate discriminator-accuracy map measured before training;
  • the gap between our method and the best erosion margin on small objects in UCAS-AOD and FAIR1M;
  • a correlation, across objects, between each object's footprint spill (mean of $1-w_r$ over its pixels at high rates) and its per-object accuracy gain over uniform rotated alignment.

A positive correlation would confirm that the method works for the reason claimed. A flat one would mean any gain comes from something else. We will also measure training-time overhead, since the per-rate weights require one strided box-filter of the mask per rate and nothing at inference.