When a detector looks at an aerial photo, a bicycle may cover about 12 pixels out of a million. Normalization layers inside the network compute an average and a spread for each feature channel over every pixel in the batch. For a tiny bicycle, those numbers are therefore set almost entirely by roofs, roads, and sea clutter. These layers are batch normalization, which rescales each channel to zero mean and unit variance using batch statistics, followed by a learned scale and shift. The same arithmetic also lets common classes such as cars outvote rare ones such as bicycles, which is the failure that class-balanced normalization was designed to fix in image classification. We propose that a detector's normalization statistics should be computed with every pixel reweighted during training. The reweighting uses the ground-truth boxes, so that background, each foreground class, and each object get a fixed share of the vote regardless of how many pixels they cover. At test time the network uses the running statistics accumulated under this weighting, so no labels and almost no extra computation are needed.
The core claim is specific and testable. On tiny-object benchmarks, background area and class frequency both distort the statistics of the shallow, high-resolution stages, which is exactly where tiny objects still exist spatially. Correcting both distortions inside those stages should improve accuracy on the smallest and rarest objects, beyond what loss reweighting and distribution-aware label assignment already give. We also expect a reversal of a known result. Class-balanced normalization has been reported to help only in the late stages of classification networks. For tiny objects, we predict the benefit moves to early stages, because late stages no longer contain the object.
Where the idea comes from and how it differs
Class-balanced normalization is established only for whole-image classification. Compound Batch Normalization replaces the single Gaussian behind batch normalization with a mixture fitted by moving-average EM, and needs an auxiliary split-normalization branch so that head classes do not capture the components. Normalizing Batch Normalization (NBN) constrains the learned scale and shift vectors to fixed norm so that the network cannot "turn down" rare-class channels. It raises ImageNet-LT tail accuracy from 6.3% to 13.4%. The same paper finds that its effect is strongest in final stages, and it names detection and segmentation as open targets. Both papers motivate the proposal. Neither ever sees a spatial grid in which one image contains thousands of background pixels and a handful of object pixels.
The closest mechanism is ProtoNorm from OIMNet++. ProtoNorm averages features into one prototype per identity before computing batch statistics, which gives each identity an equal vote. We reuse that equal-vote principle. ProtoNorm, however, operates on pooled one-dimensional region features in a late re-identification head, where the objects are already cropped. We move the principle onto two-dimensional backbone feature maps, where the dominant "majority class" is background area rather than a frequent identity.
The only spatial work that modifies normalization is DPNet. DPNet keeps a separate batch normalization for each candidate downsampling factor, because the statistics shift when resolution changes per image. That is routing by resolution. Inside each branch the statistics remain plain pixel averages, so DPNet addresses neither area imbalance nor class imbalance.
On the tiny-object side, several methods constrain the design. LIP learns importance weights for pooling. The SSD region-magnification detector and MSR-RCNN tap shallow layers because late layers have already blurred small objects. NWD with ranking-based assignment repairs label assignment for tiny boxes. All of these change which pixels survive or supervise. None changes the statistics used to normalize them, which makes our change complementary. We will stack it on top of the strongest of these methods rather than replace them.
Core Claim and Mechanism
Consider one normalization layer at a stage with stride $s$. Each training image's ground-truth boxes are rasterized at that stride into a region label for every pixel: background, or a specific object instance. The label is the smallest box covering the pixel, and any box that shrinks below one cell is kept as one cell. Each pixel $p$ receives a weight $w_p$. The background region shares total mass $\alpha$. The foreground shares $1-\alpha$, split equally across the classes present in the batch, then equally across instances within a class, then equally across that instance's pixels. The channel statistics become
$$\mu = \sum_p w_p x_p, \qquad \sigma^2 = \sum_p w_p (x_p - \mu)^2,$$
where $x_p$ is the channel activation at pixel $p$ and the weights sum to one. Setting $\alpha$ equal to the true background fraction recovers ordinary batch normalization. Setting $\alpha = 1/2$ gives background and foreground equal say. Running averages of $\mu$ and $\sigma^2$ are kept as usual and used at inference, so inference is unchanged. Batches without foreground fall back to uniform weights.
We pair these statistics with NBN's fixed-norm scale and shift, so that the balanced statistics are not undone by the learned affine parameters. We treat the pairing as a separate ablated factor rather than assuming it helps.
The worry that this simply "amplifies rare channels" does not settle the matter in either direction. Batch normalization divides by the spread, so a channel that is rare under uniform weighting already has small variance and is amplified. What balancing changes is the reference point: the mean and spread against which foreground is measured. That is why the first part of the program is a measurement, not an assumption.
Research Program
The program begins with a diagnostic on AI-TOD-v2 and VisDrone using a standard ResNet-50 FPN detector trained with synchronized batch normalization. For every normalization layer at every stage, we measure three things. The first is the fraction of statistical mass contributed by foreground pixels, which we expect to be under one percent in early stages for AI-TOD-v2, whose mean object size is 12.7 pixels. The second is the distance between uniformly weighted and balanced statistics. The third is the per-channel learned scale, split by which classes most activate each channel. If uniform and balanced statistics barely differ, the idea is dead early and cheaply. That is a legitimate outcome, and we would report it.
If they do differ, we build the balanced layer as a drop-in replacement. We then study placement by enabling it in stage 1 only, stages 1–2, stages 3–4 only, and all stages. We separate the two ingredients: area balancing only, with all foreground pooled together, versus class balancing only, where foreground classes are equalized but background is weighted by area. We also sweep $\alpha$. As a secondary question, we test whether the layer composes with DPNet's per-resolution branches by applying balanced statistics inside each branch. This checks whether routing and balancing are independent fixes.
Baselines
The naive reference is the same detector with ordinary synchronized batch normalization, identical data, schedule, and augmentation. On top of it we also run frozen ImageNet statistics and group normalization, since these are what practitioners actually use in detection. Every configuration shares NWD with ranking-based assignment as its label assignment. This ensures that measured gains come from normalization and not from assignment.
The strongest existing competitor on AI-TOD-v2 is NWD-RKA embedded in DetectoRS, which reports 24.7 AP. We will reproduce it with the authors' settings. We will add our layer with no additional tuning budget beyond a three-value sweep of $\alpha$, and give the baseline an equal three-configuration sweep of its learning rate.
On VisDrone, the file reports mAP 29.02 for YOLOv12m and 28.74 for DPNet-MobileNet-v2 in the DPNet paper. The VisDrone-DET2019 report gives a 29.62% AP ensemble ceiling and a gap of roughly 51% car AP against roughly 13% bicycle AP. We use DPNet as the dynamic-resolution baseline and our own reproduced number as the comparison point. On TinyPerson, the reference is Monotone Scale Match pretraining at 47.29 AP50 from Scale Match. No reported result was found on Lacuna for class-balanced normalization applied to any detection benchmark.
NBN applied unchanged, meaning fixed-norm affine parameters with uniform statistics, is its own baseline. It answers whether the affine fix alone suffices. A loss-level baseline combines focal loss with class-frequency repeat sampling.
The ablation that isolates the key ingredient is a shuffled-box control. Weights are computed from boxes randomly relocated within the same image, which preserves the weighting geometry and the label counts but destroys the correspondence with objects. If the shuffled control gains as much as the real one, the effect is regularization and not balancing.
We count the method as beating the baseline if it improves overall AP by at least one point over the reproduced NWD-RKA on AI-TOD-v2, averaged over three seeds with non-overlapping spread. The gain must concentrate in very-tiny AP and in the rarest VisDrone classes (bicycle, awning-tricycle, tricycle), and the shuffled control must show no comparable gain.
Why not just…?
Why not just freeze batch normalization or switch to group normalization, as most detectors already do? Frozen statistics come from ImageNet, whose objects are large and centered. The Scale Match results show that this scale mismatch costs tiny-object accuracy. Group normalization still averages uniformly over each image's spatial grid, so background dominates it in the same way. This is the most likely practical threat, so both variants appear as baselines. If either matches the balanced layer, we will report that the problem is not the statistics.
Why not just reweight the loss or resample rare classes? NBN's analysis argues that loss reweighting cannot stop the network from shrinking rare-channel scales. Its tail gain on ImageNet-LT stacked with loss-based and multi-expert methods, but that evidence comes from classification. Whether it transfers is exactly what our focal-loss-plus-repeat-sampling baseline tests, together with that baseline combined with our layer. If the layer adds nothing on top, the loss-level fix is sufficient.
Why not just upscale or tile the images? Upscaling makes tiny objects bigger. It also multiplies background area and computation, and DPNet shows that avoiding excess resolution saves over 25% of FLOPs without losing accuracy. More to the point, upscaling does not change the area ratio that dominates the statistics. We run a 2× tiled baseline at matched FLOPs to test whether resolution alone closes the gap.
Why not just apply NBN as published, in the last stages? This is the direct test of the placement reversal. If late-stage NBN with uniform statistics equals our early-stage balanced layer, the spatial ingredient is unnecessary.
Decisive Checks
Three results decide the proposal. The first is the diagnostic: balanced and uniform statistics must differ meaningfully in early stages. The second is the placement sweep: early-stage balancing should help very-tiny AP more than late-stage balancing, reversing the NBN finding for classification. The third is the shuffled-box control, which must fail. Per-class AP on VisDrone and very-tiny AP on AI-TOD-v2 are the primary metrics. We also report added training time, which should be under a few percent because the weights come from rasterized boxes, and unchanged inference cost.
References
- Compound Batch Normalization for Long-tailed Image Classification
- Normalizing Batch Normalization for Long-Tailed Recognition
- OIMNet++: Prototypical Normalization and Localization-Aware Learning for Person Search
- DPNet: Dynamic Pooling Network for Tiny Object Detection
- LIP: Local Importance-Based Pooling
- Single Shot Multibox Detector With Deconvolutional Region Magnification Procedure
- MSR-RCNN: A Multi-Class Crop Pest Detection Network Based on a Multi-Scale Super-Resolution Feature Enhancement Module
- Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark
- VisDrone-DET2019: The Vision Meets Drone Object Detection in Image Challenge Results
- Scale Match for Tiny Person Detection