Detectors lose objects of 8 to 16 pixels mainly because their own internal downsampling layers (strided convolutions and pooling) throw away the fine edges and textures that separate a car from a rooftop vent. The usual fix borrows from image super-resolution. Researchers add a module that reconstructs or regenerates the missing high-frequency detail inside the network, and it stays in the network at test time. Detection of tiny objects poses a different problem from super-resolution, though. The detail destroyed by an internal downsampling layer is not missing from the data, because it was in the input image one layer earlier. The network needs to learn to keep it, which is a teaching problem rather than a generation problem. Only part of that detail should be kept anyway: SET showed that high-frequency background texture actively hurts tiny-object detection.
We propose a training-only loss that teaches each downsampling stage of a standard detector to carry, packed into its channels, the high-frequency content of foreground objects that its coarser spatial grid can no longer represent. During training, it also pushes background high frequencies down. The target comes from the same detector run on a 2× upsampled input, with its receptive field realigned. The frequency cutoff is set by the stride of each layer rather than tuned by hand. A small decoder needed to compute the loss is discarded after training, so inference costs nothing extra. If the claim holds, recovering texture for tiny objects needs no generative model at test time. If a generative prior still adds accuracy in a controlled sensor-degradation setting, we will have located exactly where synthesis earns its cost.
Core Claim
When high-frequency evidence is lost inside the network rather than at the sensor, three choices recover most of what inference-time texture modules recover, at zero inference cost. The supervision acts on the band above each layer's Nyquist limit, which is the highest spatial frequency its output grid can represent. It applies only to foreground. It comes from a teacher whose receptive field is aligned with the student's. Uniform high-frequency enhancement, generative feature synthesis, and image-level super-resolution are each dominated by this loss on aerial tiny-object benchmarks, or match it only at several times the compute.
Where This Sits
Inference-time modules that restore high frequencies define the competition. TRD recasts downsampling as feature super-resolution. It upsamples the downsampled map, subtracts it from the input to expose lost detail, and learns that residual with convolutions. HS-FPN applies a high-pass filter with a cutoff $\alpha$ to pyramid features before fusing them. MSR-RCNN adds deconvolution-based feature super-resolution for pests. All three treat the feature map uniformly and pay their cost at inference. HS-FPN's authors note that their filter amplifies background clutter and that $\alpha$ must be re-tuned when target scale or sensor changes. We share their diagnosis that downsampling erases tiny-object texture. We differ in three ways: the recovery is learned as a training signal, it is gated by region, and the cutoff follows from the stride.
Two papers constrain the design. SET provides the motivating tension. Suppressing background high frequencies during training, using ground-truth boxes, improved AI-TOD by 3.2 AP over the prior best with zero inference overhead. That result contradicts any scheme that enhances high frequencies everywhere, so our loss must be region-selective. SET only removes background detail and never teaches the network to keep foreground detail, which leaves half of the problem untouched.
Noh et al. supply the second constraint and part of the method. They found that making low-resolution features imitate high-resolution features made small-object detection worse unless the high-resolution teacher's receptive field was dilated to cover the same relative context. We reuse their atrous alignment. Unlike them, we use no GAN, no second predictor, and no generator at inference.
LIP shows that learned, content-aware downsampling helps small objects, raising COCO $AP_s$ from 21.9% to 24.0% with Faster R-CNN. We treat it as a strong architectural competitor and as a possible student backbone. The texture-synthesis literature, including FKP, stochastic microtexture super-resolution and FLAIR, argues that deterministic upsampling over-smooths and that real texture needs a generative prior. We accept that argument for information that has truly been lost. We dispute it for information still present in the input, and one of our experiments separates the two cases.
Research Program
Take a detector whose backbone stage $s$ downsamples a feature map by stride 2. The student $S$ is the ordinary detector on the native image. The teacher $T$ is a frozen copy of the same trained detector run on the image upsampled 2×. Its stage-$s$ map $F_T$ therefore has twice the spatial resolution of the student's map $F_S$ over the same image region. Following Noh et al., the teacher's stage-$s$ convolutions use dilation 2, so each teacher unit covers the same image context as the matching student unit. Without this, the target would describe a smaller, context-poor patch.
A pixel-shuffle decoder $g$ unpacks $F_S$ into a map $g(F_S)$ at the teacher's resolution. It is a single 1×1 convolution followed by a space-to-depth inversion, and it is used only during training. The fixed operator $H$ is a high-pass filter whose cutoff equals the Nyquist frequency of the student's grid. It keeps exactly the band that the student's spatial grid can no longer represent and must therefore carry in its channels. Because this cutoff comes from the stride, there is no $\alpha$ to tune. The loss at stage $s$ is
$$ \mathcal{L}_s = \big\| M \odot \big(H(g(F_S)) - H(F_T)\big) \big\|_1 + \lambda \,\big\| (1-M) \odot H(g(F_S)) \big\|_1 , $$
where $M$ is a soft foreground mask rendered from ground-truth boxes, dilated by one stride, and $\lambda$ weights the background suppression term. The first term asks the student to encode the teacher's above-Nyquist foreground detail. The second, adapted from SET's background smoothing, penalizes above-Nyquist energy in the background. We add $\mathcal{L}_s$ to the detection loss at the two highest-resolution downsampling stages, where 8–16 pixel objects die. After training, $g$ is discarded and the deployed detector is unchanged. Boxes are needed only at training, the same supervision SET already assumes.
Before training anything, we run a cheap diagnostic that checks the premise. For a trained baseline, we compute the foreground-versus-background spectral energy at each stage on AI-TOD. We also train a linear probe that classifies tiny objects from each stage's features, and compare it with the same probe on the teacher. If probe accuracy collapses at a particular downsampling stage while the teacher's stays high, the information is lost there and is recoverable, which justifies the method. If the student's probe is already as good as the teacher's, the loss cannot help and the project should be redirected toward label assignment.
Baselines
The naive reference point is the unmodified detector on native images. We use Cascade R-CNN with ResNet-50 and FPN, for which HS-FPN reports 20.2 AP on AI-TOD, and DINO-5scale as a strong transformer detector. Every method is trained with the same schedule, augmentation, and input size, and gets the same hyperparameter search of eight configurations.
The strongest existing training-only competitor is SET, which reports 26.6 AP on AI-TOD with DINO-5scale. It is closest in spirit, since it also costs nothing at inference and also gates by region. We run it from the published modules on the same DINO-5scale setup. Beating it means a gain of at least 1 AP on top of it when combined with our loss, or a higher number when ours is used alone. Each claim must hold across three seeds.
The strongest inference-time competitor is HS-FPN, reported at 23.6 AP on AI-TOD with Cascade R-CNN and ResNet-50. On VisDrone-DET we use TRD, which reports +3.2 AP over its baseline; no absolute AP for TRD is given in the file. For both we report AP alongside FLOPs and latency. Matching their AP at zero added inference cost counts as a win. Exceeding them is a stronger win.
The NWD-RKA label assignment, at 24.7 AP on AI-TOD-v2 with DetectoRS, addresses a different failure: IoU's sensitivity to small pixel shifts on tiny boxes. We stack our loss on top of it rather than compete with it, and expect the gains to add.
Four ablations isolate the key ingredients. Removing the mask, so that high frequencies are enhanced uniformly, tests the HS-FPN/SET tension directly. Removing the high-pass filter $H$, so that the student matches all bands, reduces our method to plain feature distillation. Removing the dilation tests Noh et al.'s alignment result. Replacing the stride-derived cutoff with HS-FPN's $\alpha = 0.25$ tests whether the derived cutoff matters.
Why not just…?
Why not just feed 2× upsampled images to the detector? This is the real threat, and the teacher in our method is exactly this model. It will likely be the upper bound on accuracy, at roughly 4× the backbone compute. The question is how much of its gain we keep at 1× cost. We report the fraction of the teacher-over-baseline gap that the student closes. Closing less than half would mean the loss is not worth its training complexity, and a reader should then simply upsample.
Why not just distill features from a high-resolution teacher, without frequency bands or masks? The "remove $H$, remove $M$" ablation is precisely this. Noh et al. found that unaligned imitation of high-resolution features hurt small objects. SET's result predicts that copying background high frequencies hurts as well. If plain distillation with dilation matches our full loss, the band and mask design is unnecessary, and we will say so.
Why not just super-resolve the image with an off-the-shelf model first? Methods with generative priors carry heavy costs. DIP-FKP takes about 280 seconds per 1000×1000 image according to FKP. The GAN-based RFB-ESRGAN trades 24.03 dB PSNR against RCAN's 25.90 dB to gain perceptual sharpness, which risks invented edges on objects a few pixels wide. We test a fast perceptual super-resolution model as a preprocessing step. We expect it to sit between the baseline and the 2× teacher at higher cost and with more false positives on background texture.
Why not just use learned pooling like LIP? LIP changes the architecture and helps small objects, but it can only re-weight what lies inside each window. It does not ask the network to keep above-Nyquist detail. We train LIP-ResNet-50 as a student with and without our loss. Additive gains would mean the two methods are complementary. No gain on top of LIP would mean LIP already captures the effect.
Proposed Experiments
The main benchmarks are AI-TOD, AI-TOD-v2, and VisDrone-DET, with scale-stratified AP. COCO 2017 checks that medium and large objects do not regress, since SET's frequency-disparity result warns that suppressing high frequencies can harm large objects.
The decisive check for the "teach, don't generate" thesis is a two-regime study. In the internal-loss regime, which uses native AI-TOD images, we compare our loss against a feature-level generative module in the style of Noh et al. trained at matched compute. In the sensor-loss regime, AI-TOD images are degraded by 2× with anisotropic Gaussian blur and downsampling, following FKP's kernel family, so detail is truly absent from the input. There we compare the same pair again. The thesis predicts that our loss matches or beats generation in the first regime. In the second regime generation may finally help, and only there. A generative module winning in the first regime would refute the thesis.
We also measure the spectral energy of foreground and background at each stage after training. This verifies the mechanism directly: foreground above-Nyquist energy should rise and background energy should fall, rather than both moving together. We report AP gains together with inference FLOPs throughout, because the case for this method rests on keeping inference cost at zero.
References
- SET: Spectral Enhancement for Tiny Object Detection
- A Texture Reconstructive Downsampling for Multi-Scale Object Detection in UAV Remote-Sensing Images
- HS-FPN: High Frequency and Spatial Perception FPN for Tiny Object Detection
- MSR-RCNN: A Multi-Class Crop Pest Detection Network Based on a Multi-Scale Super-Resolution Feature Enhancement Module
- Better to Follow, Follow to Be Better: Towards Precise Supervision of Feature Super-Resolution for Small Object Detection
- LIP: Local Importance-Based Pooling
- Flow-Based Kernel Prior With Application to Blind Super-Resolution
- Stochastic Super-resolution for Gaussian Microtextures
- Solving Inverse Problems with FLAIR
- Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark
- Perceptual Extreme Super Resolution Network with Receptive Field Block