Borrowed Affinity: Soft Contrast from an Independent Teacher for Weakly Supervised Aerial Detection

Detectors trained only from image-level tags on satellite imagery, meaning labels such as "this tile contains a ship" with no boxes, learn what is present far better than where it is. Two failures dominate. In the first, predicted boxes shrink onto the most distinctive part of an object, such as an aircraft's fuselage. In the second, they spread into scenery that always appears alongside the object, such as the water around a ship or the apron around a parked plane. The best image-level method on the DIOR aerial benchmark reaches only 28.3% mAP. Fully supervised detectors on the same data reach 66.1%. This proposal adds a training signal that tells the detector which pixels belong together. That signal comes from a frozen network trained without labels, so it does not inherit the classifier's own biases. The detector is then trained with a contrastive loss whose pull between two pixels is a continuous weight set by that outside network's similarity, rather than a hard positive-or-negative label.

The obvious version of this idea is weaker. It would weight a contrastive loss by the classifier's own activation-map confidence. Because those confidences are produced by the same network that collapses onto parts and bleeds into context, weighting by them mostly re-teaches the network its own mistakes. We argue that the source of the soft weights matters more than whether they are soft. The central experiment of this proposal is built to test exactly that.

Core Claim

A dense contrastive loss, added as an auxiliary term to a standard weakly supervised aerial detector, will narrow the localization gap when its pairwise weights come from an independent label-free teacher. It will not do so when the weights come from the detector's own class activation maps (CAMs: heatmaps showing which pixels drove the classification). Concretely, we expect the independent-teacher version to improve both CorLoc and the ratio of predicted to true box area over the strongest image-level baseline. We expect the self-derived soft version to perform about the same as binary contrast.

The method draws on soft-label contrastive learning. CWCL takes similarities from a frozen teacher in one modality and uses them as continuous targets for a student. Graded similarity place recognition replaces binary pair labels with overlap scores in $[0,1]$ and reports faster convergence and better recall. Both work on whole images: zero-shot classification in one case, image retrieval in the other. We move the teacher-supplied continuous weight down to individual pixels inside an image and to the problem of finding object extent. Neither paper addresses that problem.

What Prior Work Leaves Open

The closest existing work is a weakly supervised ROI extraction method for satellite residential areas. It turns multiscale Grad-CAM maps into hard foreground and background masks. It then trains a segmentation network with a binary InfoNCE loss that pulls foreground pixels together and pushes them away from background. On 259 GeoEye-1 images this raises the F-score from 0.840 to 0.890. That paper is the motivating evidence that contrast helps here, but it fixes the pair structure from the CAM itself. Our proposal differs in two ways: pair weights are continuous, and they come from outside the classifier. We also target multi-class detection on DIOR and NWPU VHR-10.v2, not binary masks.

Aerial weakly supervised object detection (WSOD) methods constrain the design. FCC-Net co-trains a weak detector with a Faster R-CNN on hard pseudo-boxes and reaches 18.3% mAP on DIOR. Its hardest classes are context-heavy ones such as bridge and harbor. RINet enforces agreement between original and rotated views and spreads high-confidence labels to lower-scoring proposals over a proposal graph. It reaches 28.3% mAP and 52.8% CorLoc on DIOR. Both methods decide pseudo-labels with thresholds and contain no representation-level contrast. We build on RINet rather than replace it.

On the method side, Weakly Supervised Contrastive Learning shows that pulling together nearest-neighbour images avoids pushing same-class samples apart. It works on global vectors only, and we reuse its idea of a separate projection head. Soft Proposal Networks run a random walk over feature maps to produce soft proposal maps without any learning signal beyond classification. That makes them a natural training-free competitor, discussed below.

Research Program

We add a dense embedding head to a RINet-style detector backbone. During training we compute a frozen teacher's feature map on the same image. The teacher is a self-supervised encoder trained without labels; its candidates are described below. From the teacher's features we form pairwise affinities between pixel positions $i$ and $j$:

$$a_{ij} = \max(0, \cos(t_i, t_j))^{\gamma},$$

where $t_i$ is the teacher feature at position $i$ and $\gamma$ is a sharpening exponent tuned on a validation split.

We restrict anchors to a seed set $S$ of positions where the detector's current class score for a tagged class exceeds a loose threshold. The seeds only say "something of class $c$ is around here". The teacher decides how far that region extends. Each anchor $i \in S$ has continuous targets $w_{ij} = a_{ij} / \sum_k a_{ik}$ over candidate positions $j$, drawn from the same image and from other images in the batch that share the tag. The loss is

$$\mathcal{L}_{\text{soft}} = -\sum_{i \in S}\sum_{j} w_{ij}\,\log \frac{\exp(z_i^\top z_j/\tau)}{\sum_{k \neq i}\exp(z_i^\top z_k/\tau)},$$

where $z$ are the student's normalized embeddings and $\tau$ is a temperature.

The intended effect works as follows. A wing pixel resembles the fuselage in the teacher's view, so it is pulled toward the seed even though the classifier scored it low. This counters collapse onto the most distinctive part. Apron or water pixels are dissimilar in the teacher's view, so they receive almost no pull even though they co-occur with the object. This counters background bleed. The improved backbone features feed RINet's unchanged proposal scoring and refinement. We compute teacher affinities on both the original and rotated views and average them, which keeps the signal consistent with RINet's rotation branch.

Teacher choice is itself a research question, because natural-image priors are known to transfer poorly to top-down imagery (DIOR survey). We compare three teachers. The first is ImageNet-pretrained. The second is self-supervised on the unlabeled DIOR and NWPU training images. The third is an off-the-shelf remote sensing contrastive encoder in the style of Seasonal Contrast or pixel-wise multimodal contrast. The seasonal-contrast encoder is trained on far coarser imagery than DIOR's 0.5 m tiles, so we treat it as a stress test rather than a default.

Baselines

The naive reference point is OICR trained on image-level tags with the same backbone. FCC-Net reports 16.5% mAP for OICR on DIOR, and RINet reports 34.5% on NWPU VHR-10.v2. This shows what tags alone give before any localization repair.

The strongest existing competitor is RINet. It is the best image-level method in the file, and it already addresses part collapse through graph-based label spreading, so any gain over it must come from something RINet lacks. Its reported results are 28.3% mAP and 52.8% CorLoc on DIOR and 70.4% mAP on NWPU VHR-10.v2.

We will re-run RINet ourselves under three controls. First, it uses the same backbone. Second, it is initialized from the same teacher weights our student sees, so a better starting point cannot explain the gain. Third, it gets the same number of training epochs and the same hyperparameter search budget as our method. Beating RINet means at least two mAP points on DIOR plus a CorLoc gain, averaged over three seeds, with the gap exceeding seed variance. FCC-Net's reported 18.3% DIOR mAP is a secondary reference.

Three ablations isolate the key ingredient, and all keep the RINet base, schedule and seed set fixed. The first replaces $a_{ij}$ with the student's own CAM confidences, multiplying the anchor and partner scores. This is the obvious soft-label idea. The second replaces $w_{ij}$ with binary foreground and background labels from a thresholded CAM, which reproduces the ROI extraction method inside a detector. The third binarizes the teacher affinity at its median, which separates the effect of continuous weights from the effect of the teacher as a source.

On GeoEye-1 we also compare directly against the published binary-InfoNCE result of 0.964 overall accuracy and 0.890 F-score. Here our loss is plugged into their UNet setup in place of the hard-mask contrast.

Why not just…?

Why not just threshold Grad-CAM with Otsu's method? It works for isolated, distinct objects: AlexNet-WSL reaches 97.77% accuracy on WSADD. The same authors note that it fails on dense clusters and complex backgrounds. On GeoEye-1, the binary-contrast paper reports a plain Grad-CAM F-score of 0.840, below its contrast-refined result. Thresholding cannot add information the heatmap lacks, and the heatmap is exactly what collapses onto parts. We include CAM with Otsu thresholding on GeoEye-1 as a floor.

Why not just propagate CAM over the teacher's affinities at test time, with no training? This is the most serious threat. A random walk over teacher affinities, in the spirit of Soft Proposal Networks, might deliver most of the extent correction for free. We will run it with the same teacher and the same seeds, using the propagated map to rescore RINet's proposals.

If training-time contrast gives no gain over test-time propagation, the honest conclusion is that the teacher's affinity is what matters, not the loss. The proposal then becomes a study of teacher choice. We expect training to help for two reasons, though this is a hypothesis. First, the contrast also shapes cross-image class structure, which propagation within one image cannot do. Second, the learned features remove the need to run the teacher at inference.

Why not just weight the existing pseudo-label losses by confidence, or fine-tune a self-supervised remote sensing encoder? Confidence-weighted cross-entropy and the self-derived soft-contrast ablation both draw on the classifier's own scores. They test the same hypothesis, and we predict both will plateau near the binary version. Fine-tuning a pretrained encoder corresponds to our "RINet initialized from teacher weights" baseline. If that baseline matches our method, then the label-free pretraining, not the affinity loss, is the active ingredient.

Why not just collect point clicks? Wholly-WOOD reaches 62.63% AP50 on DOTA-v1.0 from points at about 1.07 s per instance. No image-level method is near that. If points are affordable, they win, and we do not claim otherwise. Our setting is archives where only scene-level or catalog tags exist, and per-object clicks across large unlabeled regions are not available. To make the trade-off explicit, we report our image-level result on DOTA-v1.0 alongside the published point result as a cost-versus-accuracy reference, not as a head-to-head contest.

Proposed Experiments and Decisive Checks

The main benchmarks are DIOR and NWPU VHR-10.v2 with image-level tags derived from the box annotations. We report mAP at IoU 0.5 and CorLoc, with GeoEye-1 as the direct comparison to binary contrast. Beyond the headline numbers, we report per-class AP on the context-heavy classes FCC-Net flags (bridge, harbor, overpass) and on ships and airplanes. We also report the median ratio of predicted to true box area for correctly classified detections, which directly measures part collapse (ratio well below one) versus background bleed (ratio well above one).

The decisive check is the comparison between the teacher-affinity and self-CAM soft weights. If the independent teacher helps and the self-derived weights do not, the core claim holds. If both help equally, softness is the active ingredient and independence is not, which falsifies the central claim. If neither beats binary contrast, the soft-label direction does not transfer to dense localization in this regime.

A second check sorts teacher quality. We predict a monotone relationship between how well a teacher's affinities match ground-truth object masks and downstream CorLoc. We measure that match on a held-out labeled subset using affinity-mask agreement. If this relationship appears, teacher choice becomes a cheap, label-light way to predict which pretrained encoder to use before running any WSOD training.