Detectors for tiny objects, such as a drone 8 pixels wide in a 1080p frame or a car 12 pixels long seen from a high-altitude camera, fail in a specific way. Deep networks shrink the image in stages to build up context. A target that small is averaged away before the network's later stages ever see it. Temporal filters, meaning trackers and persistence rules that keep only hypotheses which survive across frames, are the usual remedy for noisy detections. They act too late, though. They can only filter boxes the detector already produced, and a target erased by downsampling produces no box at all. The record of one strong tracking pipeline shows the cost. GIAOTracker reaches 54.18 mAP on VisDrone 2021 with its own detector and 82.97 mAP with ground-truth detections (GIAOTracker). Almost 29 points are lost before temporal reasoning begins.
We propose to move temporal persistence upstream, from filtering outputs to steering the network's own downsampling. The network keeps a running, camera-motion-compensated map of where weak, object-like evidence has kept reappearing over recent frames. It uses that map to decide, at every downsampling layer, which locations keep their fine detail. A real target leaves faint evidence in the same physical place frame after frame, while glints and noise do not. Persistence is therefore a cheap, physically grounded signal for where full resolution is worth spending. Stated in two sentences: the detector remembers where faint evidence has been persistent, and refuses to pool those spots away. Temporal consistency stops being a post-hoc filter and becomes the importance signal for resolution-preserving pooling.
Core Claim
Our claim concerns a specific failure. For targets under roughly 16 pixels, most missed detections are lost inside the backbone, not at the score threshold. Single-frame learned pooling cannot reliably tell a faint target from background texture of similar energy. A persistence map accumulated from shallow, high-resolution features over motion-compensated frames can. Feeding that map into a learned pooling layer should therefore raise recall on tiny targets more than either ingredient alone. It should also beat tracker-side fixes at matched compute. It should help most on cases that motion cues miss entirely, such as hovering drones, and should add no fixed confirmation delay, because the map is updated recurrently rather than over a waiting window.
Relation to Prior Work
Three systems sit closest to this intersection. All three keep spatial processing and temporal reasoning in separate modules.
GLAD is the nearest in spirit. It pairs a downsampled global detector, which uses motion found by aligning consecutive frames with a perspective transform, with a native-resolution 300×300 crop steered by a Kalman filter. It reverts to global search after 30 consecutive misses. Temporal information there decides where to place one crop through a hard-coded switch. Candidates come from frame differencing, so a hovering drone vanishes into the background. Our design differs in three ways: persistence is a soft, per-location field rather than one window; it accumulates appearance evidence rather than motion, so stationary targets still build it up; and it acts inside every pooling layer, so many targets are handled in one pass.
Dist-Tracker supplies part of our baseline. It uses a Wasserstein-distance box loss in the detector and a tracker that combines IoU with centroid distance, so association survives zero box overlap. GIAOTracker adds a Kalman filter whose measurement noise depends on detection confidence, plus an offline transformer that links fragmented tracks. Both are strong once the detector fires. GIAOTracker's own gap between predicted and ground-truth detections is the evidence that motivates working upstream.
On the spatial side, we reuse the method of LIP. LIP replaces fixed pooling with a learned importance-weighted sum and raises small-object AP on COCO from 21.9% to 24.0%. Its importance map, however, is computed from a single frame, which is exactly where faint targets are indistinguishable from texture, as ContextMix and S4ND describe.
NWD constrains the design. It shows IoU collapsing from 0.53 to 0.06 under a 1-pixel shift of a 6×6 box. Any persistence signal therefore has to tolerate small misalignments; it cannot require exact pixel overlap.
ByteTrack recovers low-score boxes that agree with a predicted trajectory, and it also motivates the work. It cannot start a track from low-score boxes at all, which is the "initialization blindness" our method targets.
Video super-resolution work that pairs degradation estimation with temporal consistency is aimed at reconstructing pixels, not at detection under a compute budget. This includes Spatial Degradation-Aware and Temporal Consistent Diffusion and Degradation-Aware Self-Supervised Multi-Temporal Super-Resolution. RestoreDet and HS-FPN address the resolution problem with single-frame static detectors.
Research Program
We will build a recurrent detector around one shared object, the persistence map $P_t$. It is stored at stride 4, before the deeper layers erase small targets. At each frame we estimate a homography $H_t$ from sparse feature matches with RANSAC, the same camera-motion compensation GIAOTracker and GLAD already use. A lightweight head on the stride-4 features produces a per-pixel evidence map $e_t \in [0,1]$. The map is then updated as
$$P_t = \lambda \, \mathcal{D}\big(\mathcal{W}_{H_t}(P_{t-1})\big) + (1-\lambda)\, e_t,$$
where $\mathcal{W}_{H_t}$ warps the previous map into the current frame, $\mathcal{D}$ is a small learned blur that allows a few pixels of unmodeled motion (so persistence tolerates the misalignments NWD warns about), and $\lambda \in (0,1)$ is a learned decay. Because the update is recurrent, a strong first sighting can be detected immediately. Persistence only raises the weight of weak evidence; it never imposes a fixed confirmation delay.
At every downsampling layer we replace standard pooling with LIP-style weighted pooling. The importance logit there sees both the local features $x$ and the persistence map resized to that layer, $\tilde P_t$:
$$y = \frac{\sum_{i \in \Omega} \exp\big(g(x_i) + \beta \tilde P_{t,i}\big)\, x_i}{\sum_{i \in \Omega} \exp\big(g(x_i) + \beta \tilde P_{t,i}\big)}.$$
Here $\Omega$ is the pooling window, $g$ is LIP's single-frame logit module, and $\beta$ is a learned per-layer gain. Persistent locations therefore dominate their pooling windows and survive to deep layers.
We will also test an optional budgeted variant. It forwards the top-$k$ persistent locations as native-resolution patches to a shared lightweight head. This lets us compare the soft pooling against GLAD-style hard crops at equal compute. Training uses short clips with truncated backpropagation through time. The evidence head gets an auxiliary loss against ground-truth object centers, smoothed with Gaussians as in NWD, so the map learns "object here" rather than "texture here." Detection losses use NWD-based assignment, so improvements in label assignment are held constant across all variants.
The analysis has three parts. First, we will instrument where tiny targets are lost. We measure the activation contrast between a target and its surround at each backbone stage, with and without persistence. The claim is that persistence keeps that contrast above the background level through the stages where it otherwise collapses. Second, we will stratify by target size, apparent speed, and hovering versus moving. Third, we will measure the cost of the mechanism, including false persistence from static clutter such as rooftop antennas and parked cars. The detection head still has to reject that clutter on appearance.
Baselines
The naive reference point is the same detector backbone with standard strided pooling, run frame by frame and followed by an off-the-shelf tracker. On ARD-MAV, GLAD reports YOLOv5s at 0.56 AP. We will reproduce this with our training recipe so that every gain is measured against our own version.
The strongest existing detection method for air-to-air tiny targets is GLAD itself, which reports 0.80 AP on ARD-MAV at 146.5 FPS on an RTX 3070. We will run GLAD's released pipeline if available, or a faithful reimplementation if not, on the same splits. Our method must reach at least that throughput on the same GPU. Beating GLAD means higher AP at equal or better frame rate, with the largest margin expected on the hovering subset, where GLAD's motion module is blind by construction.
For multi-object tracking from drones, the strongest reference is GIAOTracker's 54.18 mAP on VisDrone 2021. Because that number includes offline linking, we will also compare against its online-only 36.15 mAP. Our method is online and causal, and the fair comparison is online against online. For thermal infrared multi-drone tracking, Dist-Tracker reports 77.5% MOTA on validation and 81.32% on the hidden test set of the Anti-UAV challenge track. Its SSQ detector reports 93.9% AP50 against a YOLOv12 baseline at 84.5%. Our method will use the same FLIT association on top of our detector, so any change in MOTA comes from detection alone.
The central competitor is a tracker-side fix, described in the next section. It takes ByteTrack, replaces second-stage IoU with NWD, and lets tracks start from runs of low-score boxes that persist over a short window. It uses the identical single-frame detector and identical tuning budget for score thresholds and window length.
We will also run three ablations that isolate the key ingredient. The first is single-frame LIP with the persistence input zeroed, to test whether learned pooling alone explains the gain. The second is persistence without homography warping, to test whether motion compensation matters. The third is persistence computed from temporally shuffled frames of the same clip, which keeps the extra inputs and parameters but destroys temporal consistency.
All methods share training data, input resolution, epoch count (24, the NWD convergence requirement), and a matched hyperparameter search of equal trials. Results are reported at matched FLOPs and measured latency on both a desktop GPU and a Jetson-class device.
Why not just…?
Why not just put NWD into ByteTrack's low-confidence stage and allow track initialization from persistent low-score boxes? This is the obvious idea at the intersection, and it is a real threat. It is cheap and directly fixes both the IoU collapse NWD documents and ByteTrack's inability to start tracks from weak boxes. It works only if the erased targets still appear somewhere in the detector's low-score output. We will test that directly. For each ground-truth tiny target missed at the operating threshold, we record whether any box with score above 0.01 lies within a few pixels of it. If most misses are recoverable this way, the tracker fix captures most of the gain and our contribution shrinks to the hovering and lowest-contrast cases; we will report that outcome plainly. Our hypothesis, motivated by the GIAOTracker gap and by the backbone-contrast measurements above, is that a large share of misses leave no low-score box at all.
Why not just run the detector at higher input resolution, or on overlapping tiles? This preserves pixels by brute force. Its compute grows with image area, which is the cost GLAD avoids with a single crop, and it does nothing to separate faint targets from texture of the same size. We will run a tiled or upscaled single-frame detector at the FLOPs of our method, and at twice those FLOPs, to see where the curves cross.
Why not just stack several frames as input channels, or add a 3D convolution? This also uses time, but it does so through a fixed short window. It does not compensate for camera motion unless frames are pre-aligned, and it gives the pooling layers no explicit signal about where to keep detail. We will include an aligned frame-stacking baseline with the same backbone and parameter budget. The shuffled-frame ablation further separates "temporal consistency" from "more input." If aligned stacking matches us, the conclusion is that motion-compensated temporal input matters and the persistence-to-pooling route does not, and we will say so.
Why not just run video super-resolution before detection? The super-resolution methods in the file optimize visual reconstruction at substantial cost. On a sub-10-pixel target there is little texture to recover, so hallucinated detail risks creating false targets. We treat this approach as out of scope for the edge-latency regime and will include one lightweight super-resolution front end as a sanity check, at reported latency.
Proposed Experiments
The primary benchmark is ARD-MAV, because it combines camera ego-motion, targets as small as 8×8 pixels, and a strong published reference. We will additionally annotate a hovering subset, defined by low motion of the target relative to the background after alignment. The decisive check there is recall on hovering targets. GLAD's design predicts near-zero recall from its motion module, and our method should hold recall close to that on moving targets.
VisDrone 2021 MOT tests dense, many-target scenes, where a single crop cannot scale. We report mAP alongside identity switches, and stratify recall by object size. The thermal infrared Anti-UAV track tests textureless, low-contrast blobs, where appearance alone is weakest and persistence should matter most.
LaTOT is a single-target benchmark, where MKDNet improves Super-DiMP by 2.9% in precision and 1.2% in success rate. We will use it only as a transfer check, by plugging our backbone into Super-DiMP. NPS-Drones and Drone-vs-Bird are evaluated in the GLAD paper, but no reported result was found on Lacuna for either. We will report our own numbers for all methods there without claiming comparison to published figures.
Beyond AP, mAP, and MOTA, we report three further measures. Frames-to-first-detection measures latency and tests the claim that recurrence avoids a confirmation delay. The false-positive rate on static clutter measures the main failure mode of persistence. Recall stratified by target size and apparent speed probes the known fragility of temporal priors to fast, erratic maneuvers, and the learned blur should be the component that absorbs it.
The result that would most convince us that the idea is right is a three-way comparison. If the tracker-side NWD fix closes little of the gap, the shuffled-frame ablation loses most of the gain, and the per-stage contrast analysis shows persistent targets surviving the downsampling stages where single-frame LIP loses them, then the claim holds: temporal persistence is a better guide than single-frame appearance for deciding what not to downsample.
References
- GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021
- Global-Local MAV Detection Under Challenging Conditions Based on Appearance and Motion
- Dist-Tracker: A Small Object-aware Detector and Tracker for UAV Tracking
- LIP: Local Importance-Based Pooling
- ContextMix: A context-aware data augmentation method for industrial visual inspection systems
- S4ND: Single-Shot Single-Scale Lung Nodule Detection
- A Normalized Gaussian Wasserstein Distance for Tiny Object Detection
- ByteTrack: Multi-object Tracking by Associating Every Detection Box
- Spatial Degradation-Aware and Temporal Consistent Diffusion Model for Compressed Video Super-Resolution
- Degradation-Aware Self-Supervised Multi-Temporal Super-Resolution
- RestoreDet: Degradation Equivariant Representation for Object Detection in Low Resolution Images
- HS-FPN: High Frequency and Spatial Perception FPN for Tiny Object Detection
- Tiny Object Tracking: A Large-Scale Dataset and a Baseline