VME: A Satellite Imagery Dataset and Benchmark for Detecting Vehicles in the Middle East and Beyond

Noora Al-Emadi, Ingmar Weber, Yin Yang, Ferda OfliarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces the Vehicles in the Middle East (VME) dataset to tackle geographic bias in satellite-based object detection, providing over 100,000 annotated vehicles across 54 cities. By combining VME with existing open datasets, the authors also establish CDSI (Car Detection in Satellite Imagery), the largest global benchmark of its kind. Their findings reveal that standard models fail significantly in Middle Eastern environments, but incorporating regional data drastically improves both local and global detection accuracy.

What Problem Does This Paper Address?

Computer vision models trained on satellite imagery are notoriously sensitive to environmental context. Because vehicles typically span only a few pixels (making them "tiny objects"), detectors rely heavily on surrounding features like road textures, architectural styles, and land cover to make accurate predictions.

Unfortunately, existing open-source satellite datasets suffer from severe geographic bias, focusing predominantly on North America, Europe, and China. When state-of-the-art models trained on these datasets are deployed over the Middle East, they perform poorly. The region's unique arid landscapes, distinct urban layouts, and rapid smart-city developments present a visual context completely foreign to Western-trained models.

Visual context differences between Middle Eastern cities and other global regions.

Methods

To bridge this gap, the authors developed a rigorous pipeline for data collection, annotation, and dataset harmonization:

  1. Data Sourcing & Filtering: High-resolution (30–50 cm per pixel) pan-sharpened RGB imagery was collected via Maxar’s SecureWatch platform. Large imagery snapshots were divided into $512 \times 512$ pixel tiles. Tiles containing only empty desert or uniform green spaces were manually discarded to optimize the annotation budget.
  2. High-Fidelity Annotation: The authors employed a crowdsourced team of over 6,000 workers to generate Oriented Bounding Boxes (OBB)—which capture vehicle direction—and subsequently derived standard Horizontal Bounding Boxes (HBB). A multi-stage cross-validation and expert review process ensured 95%+ annotation accuracy.
  3. The CDSI Harmonization Pipeline: Merging multiple satellite datasets is challenging because different creators use varying taxonomies (e.g., "small car" vs. "van" vs. "vehicle"). The authors designed a pipeline to standardize all annotations into the MS-COCO format and applied a strict size filter: any object larger than 400 pixels was excluded to eliminate labeling errors and resolution mismatches.

For evaluation, the authors utilized the Slicing Aided Hyper Inference (SAHI) framework—a technique specifically designed to improve tiny object detection by slicing large images into overlapping patches during inference—paired with advanced object detectors like TOOD and DINO Swin-L.

Datasets

The paper introduces two major data contributions:

  • VME (Vehicles in the Middle East): Contains 4,282 images spanning 54 cities across 12 countries (including Syria, Egypt, Qatar, and Saudi Arabia). It features 113,737 annotated objects split into three classes: cars (101,564), buses (5,327), and trucks (6,846).
  • CDSI (Car Detection in Satellite Imagery): A massive consolidated benchmark combining VME with five prominent datasets: xView, DOTA-v2.0, VEDAI, DIOR, and FAIR1M-2.0. The final dataset includes 23,250 images, 896,760 car-related objects, and 185,619 "other" small objects serving as hard negatives.
Geographical distribution of the new VME dataset compared to the widely used xView dataset.

Key Insights and Ideas

  • The "Context Penalty" is Real: Models trained on massive datasets like FAIR1M-2.0 or VEDAI achieved dismal scores (as low as 5.1% mAP50) when tested on Middle Eastern imagery. This underscores that dataset scale cannot compensate for a lack of geographic diversity.
  • Data Harmonization Requires Pruning: Simply concatenating datasets introduces noise. By analyzing car size distributions across all datasets, the authors discovered that filtering out objects $>400$ pixels effectively purged systemic labeling errors and resolution anomalies.
  • Hard Negatives Matter: To prevent models from simply classifying any tiny blob as a car, the authors grouped non-car small objects (under 400 pixels) into an "other" category. This forced the detectors to learn fine-grained structural boundaries rather than just object scale.

Key Results

  • VME Baselines: On the standalone VME dataset, the DINO Swin-L detector achieved a baseline score of 62.7% mAP50 across all categories, outperforming TOOD (58.5%).
  • The Power of Global Pre-training: When the DINO Swin-L model was trained on the fully consolidated CDSI dataset, its performance on the VME test set skyrocketed to 86.4% mAP50.
  • VME's Contribution to the Global Benchmark: To prove VME's unique value, the authors created CDSI* (CDSI excluding VME). Removing the Middle Eastern data caused global test performance to drop by 6% mAP50 for TOOD and 4.3% mAP50 for DINO, proving that regional diversity improves a model's overall robustness.

Limitations and Future Work

While the VME dataset significantly advances the field, the authors note a few regional challenges:

  • Atmospheric Interference: Climate conditions inherent to the Middle East—such as airborne dust, sandstorms, and intense atmospheric haze—frequently degrade image clarity, leading to blurry tiles and specular reflections.
  • Localization and Background Confusion: Error analysis revealed that the majority of remaining model failures stem from imperfect bounding box localization and background confusion (false positives where the model mistakes small non-vehicle infrastructure for cars). Future work will need to focus on loss functions that penalize background confusion in arid terrains.

Summary updated

Abstract

Detecting vehicles in satellite images is crucial for traffic management, urban planning, and disaster response. However, current models struggle with real-world diversity, particularly across different regions. This challenge is amplified by geographic bias in existing datasets, which often focus on specific areas and overlook regions like the Middle East. To address this gap, we present the Vehicles in the Middle East (VME) dataset, designed explicitly for vehicle detection in high-resolution satellite images from Middle Eastern countries. Sourced from Maxar, the VME dataset spans 54 cities across 12 countries, comprising over 4,000 image tiles and more than 100,000 vehicles, annotated using both manual and semi-automated methods. Additionally, we introduce the largest benchmark dataset for Car Detection in Satellite Imagery (CDSI), combining images from multiple sources to enhance global car detection. Our experiments demonstrate that models trained on existing datasets perform poorly on Middle Eastern images, while the VME dataset significantly improves detection accuracy in this region. Moreover, state-of-the-art models trained on CDSI achieve substantial improvements in global car detection.
BibTeX
@misc{alemadi2025vme,
  author = {Noora Al-Emadi and Ingmar Weber and Yin Yang and Ferda Ofli},
  title = {{VME: A Satellite Imagery Dataset and Benchmark for Detecting Vehicles in the Middle East and Beyond}},
  year = {2025},
  eprint = {2505.22353},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.22353},
}
Figure