CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Space

Shunli Wang, Shuaibing Wang, Bo Jiao, Dingkang Yang, Liuzhen Su, Peng Zhai, Chixiao Chen, Lihua Zhang

Abstract

Estimating the 6D pose of uncooperative space objects is essential for autonomous docking and debris removal. However, the unique conditions of space—including high-contrast backgrounds like Earth and stars, extreme lighting, and severe onboard resource constraints—limit the effectiveness of traditional vision models. We propose CA-SpaceNet, a novel framework that utilizes counterfactual analysis to eliminate background interference. By decoupling target features from environmental noise through causal reasoning, CA-SpaceNet achieves state-of-the-art accuracy. To meet hardware requirements, the model is optimized via 3-bit quantization and deployed on a Processing-In-Memory (PIM) FPGA accelerator, achieving significant speedups and energy efficiency.

Introduction

The 6D pose estimation of uncooperative targets (3D rotation and translation) is a cornerstone of modern space robotics. Standard terrestrial models often struggle in orbit due to high-contrast background noise from the Earth's surface or star fields, harsh lighting conditions that create extreme shadows, and strict onboard power and compute constraints. CA-SpaceNet addresses these issues by explicitly modeling the background as a confounding variable and employing hardware-efficient design.

Methods

Counterfactual Analysis (CA)

The primary innovation of CA-SpaceNet is the use of counterfactual reasoning to achieve unbiased feature extraction. The model processes the scene through two primary paths:

  • Factual Path: Extracts features from the original image, representing the "Total Effect."
  • Counterfactual Path: Imagines a scenario where the target satellite is absent. During training, ground-truth masks are used to isolate the background, allowing the model to learn the "Side Effect" of environmental noise.
  • Pseudo Counterfactual Path: Since masks are unavailable during real-world inference, a pseudo-network ($FPN_{pc}$) is trained to mimic background features directly from the original image.

By subtracting the "Side Effect" from the "Total Effect," the model isolates the Total Direct Effect (TDE)—the pure features of the satellite itself.

Conceptual diagram of the counterfactual analysis used to isolate the Total Direct Effect (TDE) by subtracting background side effects.

Network Architecture

The framework utilizes a DarkNet-53 backbone and a Feature Pyramid Network (FPN). During training, the similarity between the true counterfactual path and the pseudo-counterfactual path is maximized, ensuring the model learns to "see through" the target to understand background noise.

Diagram of the training and inference phases for CA-SpaceNet, showing the integration of the pseudo-counterfactual path.

Hardware Optimization

To facilitate onboard deployment, the authors apply LSQ-Net quantization, reducing weights and activations to 3-bit or 8-bit precision. The authors found that while the DarkNet-53 backbone can be heavily quantized with minimal loss, the "Network Head" is more sensitive, suggesting a tiered approach to quantization. This is paired with a Processing-In-Memory (PIM) accelerator on an FPGA, which minimizes energy consumption by reducing data movement between the processor and memory.

Experimental Results

The model was evaluated on the SwissCube and SPEED datasets.

  • Accuracy: On the SwissCube dataset, CA-SpaceNet achieved an overall accuracy of 79.39% (ADI-0.1d), outperforming the previous state-of-the-art model (WDR).
  • Robustness: Qualitative analysis confirms that CA-SpaceNet's predicted keypoints remain stable in cluttered environments, whereas competing models often drift toward background features.
  • Efficiency: 3-bit quantization reduced the model size by over 90% (from 205 MB to 19 MB). The FPGA PIM implementation achieved a latency of 5.99 ms, which is 4.4x faster than an ARM v8.2 CPU and 1.7x faster than a high-end Intel Core i7 CPU.

Limitations and Future Work

The model showed a slight performance dip in very close-up scenes, likely because large target masks erase too much background context for the counterfactual path. Future work will focus on deploying the entire quantized framework to PIM chips to further meet the demands of real-time space missions.

Summary updated

Abstract

Reliable and stable 6D pose estimation of uncooperative space objects plays an essential role in on-orbit servicing and debris removal missions. Considering that the pose estimator is sensitive to background interference, this paper proposes a counterfactual analysis framework named CASpaceNet to complete robust 6D pose estimation of the spaceborne targets under complicated background. Specifically, conventional methods are adopted to extract the features of the whole image in the factual case. In the counterfactual case, a non-existent image without the target but only the background is imagined. Side effect caused by background interference is reduced by counterfactual analysis, which leads to unbiased prediction in final results. In addition, we also carry out lowbit-width quantization for CA-SpaceNet and deploy part of the framework to a Processing-In-Memory (PIM) accelerator on FPGA. Qualitative and quantitative results demonstrate the effectiveness and efficiency of our proposed method. To our best knowledge, this paper applies causal inference and network quantization to the 6D pose estimation of space-borne targets for the first time. The code is available at https://github.com/Shunli-Wang/CA-SpaceNet.
BibTeX
@inproceedings{wang2022ca,
  author = {Shunli Wang and Shuaibing Wang and Bo Jiao and Dingkang Yang and Liuzhen Su and Peng Zhai and Chixiao Chen and Lihua Zhang},
  title = {{CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Space}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
  doi = {10.1109/iros47612.2022.9981172},
}
Figure