SpaceDet: A Large-scale Space-based Image Dataset and RSO Detection for Space Situational Awareness

Jiaping Xiao, Rangya Zhang, Yuhang Zhang, Lu Bai, Qianlei Jia, Mir FeroskhanIJCAI 2025 · DOI

TL;DR

This paper introduces SpaceDet, a novel, large-scale, and highly realistic dataset for space-based Space Situational Awareness (SSA). By simulating accurate orbital dynamics and a physical camera model with photon-level noise, SpaceDet aims to bridge the gap between current simulated datasets and the demands of real-world SSA systems, enabling more robust RSO detection and tracking.

What Problem Does This Paper Address?

Space Situational Awareness (SSA) is critical for the safe operation of satellites and space infrastructure, especially as the space environment becomes increasingly crowded with debris and active objects (Resident Space Objects or RSOs). Traditional SSA relies on ground-based radar and telescopes, which are expensive and geographically limited. Space-based SSA offers a more flexible alternative, but developing effective algorithms requires realistic training data. Existing simulated datasets often lack fidelity, failing to capture crucial aspects like realistic camera models, noise distributions, and the full range of observation conditions. This deficiency makes algorithms trained on these datasets unsuitable for real-world SSA applications, leading to a critical need for more realistic and comprehensive datasets.

Methods

The core of this work is the creation of the SpaceDet dataset, generated using a high-fidelity simulator. Key methodological aspects include:

  • Realistic Image Generation: The simulator incorporates accurate space orbit dynamics and a physical camera model. Images are generated at the photon level, accounting for various noise distributions (shot noise, sensor noise, read noise, background noise) to mimic real sensor behavior.
  • Extended Observation Window: To simulate a wider field of view, four overlapping cameras with a fixed rotation angle are used.
  • Physical Camera Model: This includes a pinhole camera model, lens distortion (Brown-Conrady model), and relative illumination adjustments for effects like vignetting.
  • Data Annotation: Ground truth labels, including object classes (LEO, MEO, GEO based on Semi-Major Axis) and 2D bounding boxes, are automatically generated from orbital and positional data. Bounding boxes are derived from bearing angles, accounting for the object's path during the 1-second exposure.
  • Image Slicing: To handle the high resolution (4418x4418 pixels) and make the dataset compatible with standard deep learning frameworks, images are sliced into smaller 260x260 pixel segments with overlap.

Datasets

The primary contribution is the introduction of the SpaceDet dataset, which is divided into three subsets:

  • SpaceDet-100: A minimal dataset for initial training and testing (100 images).
  • SpaceDet-5000: A larger subset for more rigorous training and validation (5000 images, primarily from one camera).
  • SpaceDet-full: The complete dataset, featuring 20,000 images from all four simulated cameras, totaling 781.5 GB of image data and 25.9 MB of ground truth labels.

This dataset is compared against existing benchmarks like BUAA-SID-share 1.0, SPARK, and RSONAR, highlighting SpaceDet's higher resolution and more comprehensive RSO representation.

Key Insights and Ideas

  • The Realism Gap is Critical: Existing simulated datasets, like SPARK, fail to capture the nuanced visual characteristics of space-based observations, rendering algorithms trained on them less effective in real-world SSA scenarios. SpaceDet's photon-level simulation and physical camera model directly address this.
  • High Resolution is a Double-Edged Sword: While the 4418x4418 resolution is crucial for detecting small, distant RSOs, it poses significant computational challenges for processing and storage, necessitating techniques like image slicing.
  • Sparse Data is the Norm: Over 99% of the SpaceDet images are background, with targets occupying only a few pixels. This extreme sparsity presents a unique challenge for object detection models, requiring careful dataset balancing and specialized algorithms.
  • SOTA Detectors Struggle in Space: Standard object detection models, even advanced ones like YOLOv8m and Transformer-based architectures, show significantly lower performance on SpaceDet compared to typical benchmarks, underscoring the need for SSA-specific algorithm development.

Key Results

  • Benchmarking SOTA Models: The paper evaluates several object detection models (YOLOv5, YOLOv8, YOLOv10, DETR, DINO, Faster R-CNN) on SpaceDet-100. YOLOv8m demonstrated a good balance of accuracy and speed, making it a strong candidate for SSA applications.
  • Tracking Performance: Various tracking methods were benchmarked. Feature-based tracking using YOLO features outperformed traditional methods and IoU-based approaches, achieving a higher accuracy (27% improvement over Euclidean distance).
  • Dataset Size: SpaceDet-full offers a substantial 781.5 GB of image data, providing a rich resource for developing and testing advanced SSA algorithms.

Limitations and Future Work

  • Simulation vs. Reality: While SpaceDet aims for high realism, it is still a simulation. The authors acknowledge that it may not capture all the complexities and variabilities of actual space environments.
  • Computational Demands: The high resolution and large dataset size require significant computational resources for processing and storage, potentially limiting accessibility for some researchers.
  • Limited Scenario Coverage: The dataset, while extensive, may not cover every possible SSA scenario, suggesting the need for further validation with real-world data.
  • Future Work: The authors suggest that SpaceDet can facilitate the development of new ODT algorithms specifically tailored for SSA and serve as a benchmark for evaluating advanced SSA techniques. Further research could explore more sophisticated noise models and a wider range of simulated space conditions.

Summary updated

BibTeX
@inproceedings{xiao2025spacedet,
  author = {Jiaping Xiao and Rangya Zhang and Yuhang Zhang and Lu Bai and Qianlei Jia and Mir Feroskhan},
  title = {{SpaceDet: A Large-scale Space-based Image Dataset and RSO Detection for Space Situational Awareness}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1045},
}
Figure