Clutter-Gated Sub-Pixel Peak Decoding for Tiny Targets in Single-Stage Detectors

Fast detectors built for drones and edge devices predict object centers on a coarse grid, typically one cell per 8 or 16 image pixels. For a target only a few pixels wide, such as a distant vehicle or a dim infrared blip, the detector then has to work out where inside a 40-pixel neighborhood the object actually sits. The usual fixes treat this purely as a resolution problem. They add a finer, more expensive output layer, or they estimate a fractional offset inside the grid cell. We argue that in cluttered scenes the dominant error is something else: bias from background structure. Bright rocks, cloud edges, and building corners inside the local window pull the estimated center toward themselves. A decoder that ignores this will stay inaccurate however finely it interpolates.

The proposal is to decode each detected peak by fitting a small continuous Gaussian to the detector's own learned response map. Each grid cell in the local window carries a learned "ownership" weight, so cells whose response belongs to nearby clutter are discounted. The whole decoder is differentiable and sits inside the training loss. In two sentences: sub-pixel accuracy for tiny targets is limited by clutter bias as much as by grid size, and a weighted Gaussian fit on a learned, clutter-suppressed heatmap can fix both. If it works, a cheap stride-8 detector should localize tiny targets about as well as one carrying an expensive stride-4 head. We will test that claim directly, on infrared and aerial benchmarks where exact sub-pixel ground truth can be manufactured.

Core Claim

Two terms make up the localization error of a single-stage center-point detector on tiny targets. The first is a quantization term: the fraction of a cell lost when the center is snapped to the grid. The second is a clutter-bias term: the displacement caused by non-target response inside the decoding window. Prior sub-pixel work attacks only the first term, and the second grows as the signal-to-clutter ratio falls.

We claim three things. First, below a modest signal-to-clutter ratio the clutter term dominates. Second, both existing learned remedies are biased by clutter. One is the CenterNet-style offset head that regresses a fractional shift. The other is the local soft-argmax, a differentiable center-of-mass over a small patch of the response map, used in Subpixel Heatmap Regression for Facial Landmark Localization. Third, an ownership-weighted Gaussian fit, trained end to end, removes most of that bias at negligible compute cost.

A secondary claim concerns what the extra resolution actually buys. The expensive stride-4 head of Small Object Detection in UAV Images Based on YOLOv8n raised compute from 8.1 to 12.5 GFLOPs and recall from 31.3% to 35.6%. We expect the gain to split into a recall part, which our decoder cannot replace, and a localization part, which it can. We will measure the two parts separately rather than assert that one method replaces the other.

Research Program

The detector is a CenterNet-style head on a YOLOv8n-scale backbone. It predicts a class heatmap at stride $s$ (8 by default), box size, and one extra channel: a per-cell ownership logit. Following the facial-landmark work, the heatmap is trained against Gaussians centered at the continuous ground-truth coordinates, so no quantization enters at the label stage.

At decode time we take each local maximum and a $k \times k$ window around it (with $k$ equal to 5). There we fit a Gaussian by weighted least squares on the logarithm of the heatmap. This works because the log of a Gaussian is a quadratic, so the fit has a closed form and is differentiable. Separably per axis, the fit solves

$$\min_{a,b,c}\ \sum_{i \in W} w_i\,\big(\ln h_i - a - b\,x_i - c\,x_i^2\big)^2,\qquad \hat{x} = -\frac{b}{2c},$$

where $W$ is the window, $h_i$ is the heatmap value at cell $i$, $x_i$ is that cell's coordinate, and $w_i$ is the sigmoid of the ownership logit. The estimated center is $\hat{x}$.

This is the classical parametric fit of Detection of intensity peaks in high-resolution transmission electron microscopy image based on YOLOv3, with three changes. It runs on learned features rather than raw intensities. It uses learned weights rather than uniform ones. And gradients flow through it, rather than it running as a disconnected 7.3-second-per-frame post-processing step. We supply the method from that paper and from 3-D Shape Reconstruction of Pipe with Omnidirectional Laser and Omnidirectional Camera. The same papers also constrain the design: raw-intensity Gaussian fits break on weak, diffuse, or distorted peaks. That failure is why we fit a learned map instead.

Training uses three losses. The first is the usual focal heatmap loss. The second is an L1 loss on the decoded continuous center $\hat{x}$, back-propagated through the fit into both the heatmap and the ownership map. The third is a shift-consistency term borrowed from the facial-landmark paper's Siamese training: we warp the image by a random fractional translation and require the decoded center to move by the same amount. This term matters for point-labelled infrared data, whose annotated clicks are themselves integer-quantized. Shift consistency teaches sub-pixel behavior without sub-pixel labels.

The ownership map receives no direct supervision. Its only incentive is to downweight cells that bias the decoded center. We will check whether it actually learns to track clutter by correlating it with target masks where masks exist.

We also address the window-capture failure reported in Learning to Make Keypoints Sub-pixel Accurate, where refinement fails if the coarse detection misses the window. At stride 8, a 5-cell window spans 40 image pixels, so capture failure should be rare. Clutter contamination of the window, however, becomes the norm. That tension motivates the design, and we will report capture-failure rates explicitly.

The evaluation spine is a controlled benchmark with exact sub-pixel truth. We render point-spread-function targets at known continuous centers, sized 1 to 9 pixels, onto real infrared backgrounds, and sweep the signal-to-clutter ratio. The backgrounds are NUDT-SIRST frames with targets inpainted out, plus the synthesized clutter negatives of SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection. Because we know the true center, the error decomposes exactly. Evaluating the decoder on the same target pasted onto a flat background isolates the quantization term, and the difference from the cluttered case isolates the clutter term.

Baselines

The naive reference is the same detector decoding the integer argmax of the stride-8 heatmap with no offset. Its error should sit near the uniform-quantization floor, roughly a third of a cell.

The strongest direct competitor is a CenterNet-style offset head: a regression branch predicting the fractional displacement within the peak cell, trained on the same continuous labels. It already produces sub-pixel output in a single forward pass, so it is the method our decoder must beat. The second learned competitor is the unweighted local soft-argmax of Subpixel Heatmap Regression for Facial Landmark Localization. That paper reports reducing WFLW error from 4.21 to 3.72 NME, but only on clean, cropped faces. The classical competitor is the decoupled pipeline of Detection of intensity peaks in high-resolution transmission electron microscopy image based on YOLOv3: a Gaussian fit to raw pixel intensities inside each predicted box.

All four decoders share the same backbone, training schedule, augmentations, and tuning budget, with the same number of learning-rate and loss-weight trials for each. They differ only in the decoding head.

On resolution, the baseline is YOLOv8n with the added stride-4 head from Small Object Detection in UAV Images Based on YOLOv8n. It reports 37.6% [email protected] on VisDrone at 12.5 GFLOPs. Alongside it we run SSCW-YOLO: A Lightweight and High-Precision Model for Small Object Detection in UAV Scenarios, which reports 37.8% [email protected] and 22.4% small-object [email protected] at 8.7 GFLOPs. We found no reported VisDrone [email protected] or centroid error on Lacuna for either, so we will retrain both under our protocol.

On infrared, the strongest reported detector is AEDNet: Anisotropic Enhanced Dual-Branch Network for Infrared Small Target Detection. On NUDT-SIRST it reports IoU 82.77%, detection probability 80.70%, and false-alarm rate 4.69×10⁻⁶. We found no reported centroid-localization error for it on Lacuna. We will compute one from its predicted masks, which makes it a strong segmentation-based localization reference.

Four ablations isolate the key ingredient. The first sets all ownership weights to one. The second trains without the decoded-coordinate loss, so the fit is post hoc on a learned heatmap. The third removes shift consistency. The fourth pairs our decoder with the stride-4 head, testing whether the gains stack.

Beating the offset head means lower centroid error on the synthetic benchmark at low signal-to-clutter, outside seed variance, with detection probability and false alarms unchanged. Beating the stride-4 head on localization means matching its localization error on correctly detected targets at no more than about a tenth of its added compute. We do not expect to match its recall.

Why not just…?

Why not just use the offset head that CenterNet-style detectors already have? This is the most serious threat. A regression branch sees the same features as our ownership map and could, in principle, learn to discount clutter implicitly. Our argument that it will not rests on an untested hypothesis. A single regressed vector must encode both "where is the peak" and "which nearby mass to ignore". An explicit weighted fit separates these, and its geometry keeps the estimate unbiased on clean peaks. The synthetic signal-to-clutter sweep settles the question. If the offset head's clutter term is as small as ours across the sweep, the core claim is wrong and we will report it as such.

Why not just add a stride-4 head? It clearly helps, but mostly by detecting more objects. On VisDrone, recall rose from 31.3% to 35.6% at a 54% compute increase (Small Object Detection in UAV Images Based on YOLOv8n). A finer grid shrinks the quantization term but not the clutter term, because clutter within a few pixels of a tiny target is still inside the window. Our experiments split its gain into recall and localization on matched detections. If its localization advantage over our stride-8 decoder turns out to be large, the case for our method on edge hardware weakens to "localization for free when you cannot afford the head".

Why not just fit a Gaussian to the raw pixels after detection? The paper that did exactly this reports that the fit fails on weak, diffuse, or distorted peaks. It also runs at about 7.3 seconds per 1024×1024 frame (Detection of intensity peaks in high-resolution transmission electron microscopy image based on YOLOv3). Raw intensities cannot tell a target from a bright-spot mimic. Our decoder keeps the parametric fit but applies it to a learned, weighted map, and the raw-intensity fit is included as a baseline on the same detections.

Why not just train on more clutter negatives? Negative synthesis such as that in SIRST-5K sharply reduces false alarms, reaching a false-alarm rate of 2.735×10⁻⁶. It targets whether a response exists, not where a true target's center lies when clutter sits beside it. We will train every decoder variant with and without these negatives to check whether negatives alone shrink the clutter-bias term.

Proposed Experiments

The decisive experiment is the synthetic sweep. For each decoder we plot centroid root-mean-square error in pixels against signal-to-clutter ratio, target size, and stride (4, 8, 16), split into quantization and clutter terms using the flat-background twin of each sample. We also report detection probability and false-alarm rate, so that localization gains cannot be bought by suppressing hard detections.

The prediction is falsifiable in two ways. The clutter term should exceed the quantization term for the offset head and for plain soft-argmax below a certain signal-to-clutter ratio. Our decoder should cut the clutter term substantially while leaving the quantization term near the others. If the clutter term never dominates, the premise fails.

On real infrared data we train on NUDT-SIRST, IRSTD-1K, and NUAA-SIRST under two label regimes. In the full-mask regime, the mask centroid is a genuine continuous label. In the single-point regime, the labels are integer clicks, the setting of From Easy to Hard: Progressive Active Learning Framework for Infrared Small Target Detection with Single Point Supervision. We report detection probability and false alarms alongside centroid error against mask centroids. The point-label regime tests whether shift consistency recovers sub-pixel accuracy that the labels themselves do not contain.

On VisDrone we decode box centers with each method and report [email protected], [email protected], and small-object AP. Stricter thresholds are where a fraction of a pixel matters for 8-pixel boxes. We also report GFLOPs and measured frame rate on an embedded board, against the stride-4 YOLOv8n and SSCW-YOLO.

Finally, we inspect the learned ownership maps against target masks and against the false-alarm locations from the negative set. The aim is to confirm that the weights discount clutter rather than doing something incidental.