SLIP-RS: Structured-Attribute Language-Image Pre-Training for Remote Sensing Object Detection

Chenxu Wang, Yuxuan Li, Yunheng Li, Xiang Li, Jingyuan Xia, Qibin Hou

Executive Summary

Existing language-image pre-training models for remote sensing (RS) object detection are fundamentally constrained by "Monolithic Label Learning." This traditional approach relies on exhaustively enumerating open-set categories using black-box data to acquire fine-grained representations—a dependency that is highly incompatible with the inherent data scarcity of the remote sensing domain.

To overcome this bottleneck, researchers introduce SLIP-RS, a novel framework that establishes a "Structured-Attribute Decoupling Paradigm." Instead of treating objects as indivisible monolithic labels (e.g., "Boeing 747"), SLIP-RS maps the open-ended category space into a finite, physically meaningful attribute space (e.g., "four engines + swept wings"). Supported by a statistically rigorous data engine, the authors curated RS-Attribute-15M—the largest attribute-annotated dataset in the field—and achieved unprecedented state-of-the-art performance in fine-grained detection and cross-domain generalization.

The Problem: Monolithic Label Learning

In remote sensing, distinguishing between visually similar objects—such as different classes of aircraft or maritime vessels—is a persistent challenge. Current "Open-Vocabulary" models, largely inspired by CLIP, rely on Monolithic Label Learning. They treat a category name as an indivisible point in semantic space.

This approach suffers from two critical flaws in the RS domain:

  1. Data Scarcity: Learning the subtle visual differences between a Boeing 737 and a 747 through raw text-image pairs requires massive datasets. Remote sensing data is notoriously scarce compared to general web-scale imagery.
  2. Semantic Binding: Models tend to "memorize" the entire label rather than "understanding" its constituent parts. If a model has not encountered a specific category name during training, it struggles to identify it, even if it recognizes all the individual physical components (like wings or engines).

The SLIP-RS Methodology

SLIP-RS transcends these limitations by shifting the focus from arbitrary category names to a finite, physically meaningful attribute space. This paradigm is realized through two core technical pillars.

1. Structured-Attribute Contrastive Learning (SACL)

Instead of standard image-text alignment, SACL enforces the learning of decoupled, intrinsic visual logic through combinatorial attribute augmentation:

  • Positive Prompts: During training, attributes are randomly shuffled and dropped. This forces the model to become invariant to the order of descriptions and robust to partial occlusions.
  • Counterfactual Hard Negatives: While standard models learn by comparing vastly different objects (e.g., a "plane" to a "ship"), SACL utilizes Attribute Replacement to compare near-identical objects (e.g., a "four-engine plane" to a "twin-engine plane"). This "near-manifold" negative mining forces the model to focus on fine-grained structural details rather than broad semantic strokes.

2. Conformal Attribute Reliability Engine (CARE)

To address the lack of labeled attribute data, the authors developed CARE to generate high-quality pseudo-labels. Unlike standard methods that rely on arbitrary confidence thresholds (e.g., 0.9), CARE leverages Conformal Prediction theory. This provides a rigorous statistical boundary to control the False Discovery Rate, ensuring that the generated annotations are high-fidelity and reliable despite originating from noisy sources.

RS-Attribute-15M and Evaluation Benchmarks

A major contribution of this work is RS-Attribute-15M, the largest dataset of its kind, containing over 15 million instance-level attribute annotations. It was constructed by revitalizing 14 open-source datasets (RS-Attribute-O) and scraping diverse third-party channels (RS-Attribute-C).

The model was rigorously evaluated on standard benchmarks including DOTA-v2.0 (horizontal and oriented boxes), DIOR-H, and Xview, alongside a specialized fine-grained benchmark derived from Fair1M, MAR20, and HRSC2016 to specifically test attribute recognition.

Key Insights and Results

SLIP-RS demonstrates that structured attributes are a vital foundation for scalable remote sensing models.

  • Combinatorial Reasoning Over Memorization: By learning attributes like "propeller" and "straight wing" independently, the model can recognize entirely novel categories simply by "reading" their physical parts.
  • Breaking the "Black Box": Visualizations of activation maps reveal that when queried for "Engine Count," the model specifically isolates the engines; for "Wing Shape," it highlights wing boundaries. This confirms the model learns actual geometry, not just contextual correlations.
  • State-of-the-Art Performance: On DOTA-v2.0, SLIP-RS (ConvNeXT-L) achieved 47.14% mAP, outperforming specialized models like OpenRSD.
  • Zero-Shot Prowess: When trained exclusively on the custom RS-C dataset and tested on DOTA-v2.0 (without seeing DOTA's training labels), it achieved an impressive 35.63% mAP.
  • Fine-Grained Dominance: In "Plane Purpose" recognition (e.g., distinguishing a bomber from a fighter), SLIP-RS achieved 72.04% mAP, obliterating previous state-of-the-art methods that struggled around 9%.
  • Efficiency: A lightweight ConvNeXT-T version of SLIP-RS outperformed much larger ViT-L-based models on the Xview dataset (27.2% vs 26.6% AP50).

Limitations and Future Work

While the attribute system is highly sophisticated for aircraft (covering engine layout, wing shape, etc.), the authors note it remains relatively coarse for ships and vehicles.

Future directions include expanding the attribute dictionary to encompass ship-specific features (e.g., "island position" or "mast count"), integrating universal primitives like color and scale, and exploring "compositional zero-shot detection," where users could detect completely novel objects simply by describing their attributes in a prompt.

Summary updated

Abstract

Existing language-image pre-training for remote sensing object detection is constrained by Monolithic Label Learning, which relies on exhaustively enumerating open-set categories via black-box data to acquire fine-grained representations, creating a dependency incompatible with the domain's inherent data scarcity. To transcend this bottleneck, we propose SLIP-RS, establishing a Structured-Attribute Decoupling Paradigm that maps the open-ended category space into a finite, physically meaningful attribute space, unlocking fine-grained discriminability via explicit structural logic. This paradigm is realized via two technical pillars: (1) Structured-Attribute Contrastive Learning, which enforces the learning of decoupled intrinsic visual logic via combinatorial attribute augmentation; and (2) Conformal Attribute Reliability Engine, which leverages conformal prediction theory to rigorously distill high-fidelity supervision from noisy sources, yielding RS-Attribute-15M, the largest dataset with over 15 million attribute annotations. Extensive experiments demonstrate that SLIP-RS establishes unprecedented performance in fine-grained detection and cross-domain generalization, validating structured attributes as a vital foundation for scalable remote sensing models.
BibTeX
@inproceedings{wang2026slip,
  author = {Chenxu Wang and Yuxuan Li and Yunheng Li and Xiang Li and Jingyuan Xia and Qibin Hou},
  title = {{SLIP-RS: Structured-Attribute Language-Image Pre-Training for Remote Sensing Object Detection}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=O2znfxqcIj},
}
Figure