Cross-DINO: Cross the Deep MLP and Transformer for Small Object Detection

Guiping Cao, Wenjian Huang, Xiangyuan Lan, Jianguo Zhang, Dongmei Jiang, Yaowei Wang

Abstract

Small Object Detection (SOD) remains a significant challenge for Transformer-based detectors due to feature blurring in deep layers and inherently low classification confidence for tiny targets. We present Cross-DINO, a novel architecture that integrates a deep MLP backbone with a Transformer-based detection head. By introducing a Cross Coding Twice Module (CCTM) to preserve fine-grained details and a size-aware Boost Loss to calibrate confidence scores, Cross-DINO achieves state-of-the-art performance on small object benchmarks. On the COCO dataset, it reaches 36.4% $AP_S$, a +4.4% improvement over the DINO baseline while maintaining high efficiency.

Introduction: The Challenges of Small Object Detection

Detecting small objects—such as distant pedestrians, tiny vehicles in drone imagery, or objects occupying just a few pixels—is a major hurdle in computer vision. While Transformer-based models like DETR and DINO have revolutionized general object detection, they struggle with SOD for three primary reasons:

  1. Limited Context from Backbones: Standard CNN backbones (like ResNet) focus on local features and often miss the broader, long-range contextual clues needed to identify tiny, ambiguous objects.
  2. Feature Blurring in Transformers: The multiple attention layers in a Transformer encoder tend to blur fine-grained details layer-by-layer. This is a minor issue for large objects but disastrous when an object is only a few pixels wide.
  3. Low Confidence Scores: Detectors naturally output lower classification confidence scores for small objects compared to large ones. Because bounding box prediction and classification are often treated separately, this low confidence leads to frequent missed detections.

Methodology

To bridge the gap between general object detection and SOD, Cross-DINO introduces three key innovations designed to preserve detail and boost detection confidence.

CLAP-Strip-MLP Backbone

Instead of a standard CNN, Cross-DINO uses a deep MLP (Multi-Layer Perceptron) backbone to capture both short- and long-range spatial information simultaneously. To overcome the fixed-size input limitation of traditional MLPs, the authors introduce CLAP (Cropping with Adaptive overLaPping). This technique dynamically crops and pads images into overlapping mini-patches, allowing the MLP to process arbitrary image sizes efficiently.

Cross Coding Twice Module (CCTM)

To mitigate the "feature blurring" caused by successive attention layers in the Transformer encoder, the CCTM acts as a bridge. It adaptively fuses crisp, fine-grained features from the MLP backbone directly into the blurred Transformer encoder's output using a two-step cross-gating mechanism, ensuring that tiny details are not lost during processing.

Boost Loss

The authors designed a new soft label called Category-Size (CS) and a corresponding "Boost Loss." This loss function adaptively scales up the penalty for low-confidence predictions specifically when the ground-truth bounding box is small. By linking object size directly to the classification loss, the network is forced to pay more attention to tiny objects that might otherwise be ignored as noise.

Experimental Results

Cross-DINO was evaluated across a diverse set of benchmarks, including COCO 2017, VisDrone2019, and AI-TOD. On the COCO benchmark, Cross-DINO achieved 36.4% $AP_S$ (Average Precision for small objects) with only 45M parameters. This significantly outperforms the baseline DINO model by +4.4% $AP_S$ while using fewer parameters and FLOPs.

Comparison of AP and $AP_S$ on COCO val2017 across different models and epochs.

In specialized datasets like VisDrone2019, the model improved overall AP by +3.7% compared to DINO. On the SODA-D high-resolution traffic dataset, Cross-DINO outperformed previous state-of-the-art models by +3.3% for extremely small objects.

Analysis and Insights

A critical insight of this research is that attention is not always "all you need" for tiny details; successive attention layers can actually degrade high-frequency information. Re-injecting early backbone features via the CCTM is a highly effective workaround. Furthermore, addressing the statistical correlation between object size and classification confidence is vital.

Average class prediction scores for different object sizes, comparing Cross-DINO and DINO.

As illustrated in the figure above, Cross-DINO significantly increases the average class prediction scores for small objects compared to the standard DINO framework. This improvement in confidence directly translates to higher recall and better overall detection performance for tiny targets.

Conclusion and Future Work

Cross-DINO demonstrates that deep MLPs can rival CNNs for dense prediction tasks when paired with adaptive cropping techniques. By explicitly addressing feature blurring and confidence calibration, the model sets a new benchmark for efficient small object detection. Future work will explore optimizing these end-to-end frameworks for even smaller targets, such as those found in the AI-TOD dataset, where objects average only 12.8 pixels in size.

Summary updated

BibTeX
@article{cao2025cross,
  author = {Guiping Cao and Wenjian Huang and Xiangyuan Lan and Jianguo Zhang and Dongmei Jiang and Yaowei Wang},
  title = {{Cross-DINO: Cross the Deep MLP and Transformer for Small Object Detection}},
  journal = {IEEE Trans. Multimedia},
  year = {2025},
  url = {https://openreview.net/forum?id=4OZrTBL7iV},
}
Figure