TL;DR
This paper introduces the Vehicles in the Middle East (VME) dataset to tackle geographic bias in satellite-based object detection, providing over 100,000 annotated vehicles across 54 cities. By combining VME with existing open datasets, the authors also establish CDSI (Car Detection in Satellite Imagery), the largest global benchmark of its kind. Their findings reveal that standard models fail significantly in Middle Eastern environments, but incorporating regional data drastically improves both local and global detection accuracy.
What Problem Does This Paper Address?
Computer vision models trained on satellite imagery are notoriously sensitive to environmental context. Because vehicles typically span only a few pixels (making them "tiny objects"), detectors rely heavily on surrounding features like road textures, architectural styles, and land cover to make accurate predictions.
Unfortunately, existing open-source satellite datasets suffer from severe geographic bias, focusing predominantly on North America, Europe, and China. When state-of-the-art models trained on these datasets are deployed over the Middle East, they perform poorly. The region's unique arid landscapes, distinct urban layouts, and rapid smart-city developments present a visual context completely foreign to Western-trained models.
Methods
To bridge this gap, the authors developed a rigorous pipeline for data collection, annotation, and dataset harmonization:
- Data Sourcing & Filtering: High-resolution (30–50 cm per pixel) pan-sharpened RGB imagery was collected via Maxar’s SecureWatch platform. Large imagery snapshots were divided into $512 \times 512$ pixel tiles. Tiles containing only empty desert or uniform green spaces were manually discarded to optimize the annotation budget.
- High-Fidelity Annotation: The authors employed a crowdsourced team of over 6,000 workers to generate Oriented Bounding Boxes (OBB)—which capture vehicle direction—and subsequently derived standard Horizontal Bounding Boxes (HBB). A multi-stage cross-validation and expert review process ensured 95%+ annotation accuracy.
- The CDSI Harmonization Pipeline: Merging multiple satellite datasets is challenging because different creators use varying taxonomies (e.g., "small car" vs. "van" vs. "vehicle"). The authors designed a pipeline to standardize all annotations into the MS-COCO format and applied a strict size filter: any object larger than 400 pixels was excluded to eliminate labeling errors and resolution mismatches.
For evaluation, the authors utilized the Slicing Aided Hyper Inference (SAHI) framework—a technique specifically designed to improve tiny object detection by slicing large images into overlapping patches during inference—paired with advanced object detectors like TOOD and DINO Swin-L.
Datasets
The paper introduces two major data contributions:
- VME (Vehicles in the Middle East): Contains 4,282 images spanning 54 cities across 12 countries (including Syria, Egypt, Qatar, and Saudi Arabia). It features 113,737 annotated objects split into three classes: cars (101,564), buses (5,327), and trucks (6,846).
- CDSI (Car Detection in Satellite Imagery): A massive consolidated benchmark combining VME with five prominent datasets: xView, DOTA-v2.0, VEDAI, DIOR, and FAIR1M-2.0. The final dataset includes 23,250 images, 896,760 car-related objects, and 185,619 "other" small objects serving as hard negatives.
Key Insights and Ideas
- The "Context Penalty" is Real: Models trained on massive datasets like FAIR1M-2.0 or VEDAI achieved dismal scores (as low as 5.1% mAP50) when tested on Middle Eastern imagery. This underscores that dataset scale cannot compensate for a lack of geographic diversity.
- Data Harmonization Requires Pruning: Simply concatenating datasets introduces noise. By analyzing car size distributions across all datasets, the authors discovered that filtering out objects $>400$ pixels effectively purged systemic labeling errors and resolution anomalies.
- Hard Negatives Matter: To prevent models from simply classifying any tiny blob as a car, the authors grouped non-car small objects (under 400 pixels) into an "other" category. This forced the detectors to learn fine-grained structural boundaries rather than just object scale.
Key Results
- VME Baselines: On the standalone VME dataset, the DINO Swin-L detector achieved a baseline score of 62.7% mAP50 across all categories, outperforming TOOD (58.5%).
- The Power of Global Pre-training: When the DINO Swin-L model was trained on the fully consolidated CDSI dataset, its performance on the VME test set skyrocketed to 86.4% mAP50.
- VME's Contribution to the Global Benchmark: To prove VME's unique value, the authors created
CDSI*(CDSI excluding VME). Removing the Middle Eastern data caused global test performance to drop by 6% mAP50 for TOOD and 4.3% mAP50 for DINO, proving that regional diversity improves a model's overall robustness.
Limitations and Future Work
While the VME dataset significantly advances the field, the authors note a few regional challenges:
- Atmospheric Interference: Climate conditions inherent to the Middle East—such as airborne dust, sandstorms, and intense atmospheric haze—frequently degrade image clarity, leading to blurry tiles and specular reflections.
- Localization and Background Confusion: Error analysis revealed that the majority of remaining model failures stem from imperfect bounding box localization and background confusion (false positives where the model mistakes small non-vehicle infrastructure for cars). Future work will need to focus on loss functions that penalize background confusion in arid terrains.
Summary updated