MarsSQE: Stereo Quality Enhancement for Martian Images Using Bi-level Cross-view Attention

Mai Xu, Yinglin Zhu, Qunliang Xing, Jing Yang, Xin ZouarXiv 2024 · DOI · arXiv

Abstract

High-resolution Martian imagery captured by rovers faces significant bandwidth constraints, necessitating heavy lossy compression that introduces visual artifacts. Existing quality enhancement models often fail to account for the unique unstructured terrain of Mars or the stereo nature of rover camera systems. We present MarsSQE, a novel framework utilizing a bi-level cross-view attention mechanism to exploit redundant information across stereo views. By combining patch-level and pixel-level attention, MarsSQE effectively restores fine textures in irregular Martian landscapes. Experimental results on a newly established Stereo Martian Image Dataset demonstrate that MarsSQE outperforms state-of-the-art monocular and Earth-based stereo models in both quantitative metrics and visual fidelity.

Introduction

When rovers like Perseverance or Zhurong capture high-resolution images of the Martian surface, they encounter a massive bottleneck: the 400-million-kilometer communication gap back to Earth. Due to extremely limited bandwidth, these images must be heavily compressed (usually via lossy JPEG), which introduces artifacts such as blurring, blocking, and loss of fine texture.

While many quality enhancement algorithms exist, they are typically designed for Earth-based, monocular photos. Mars rovers, however, use stereo cameras for navigation and depth sensing. Existing methods fail to exploit the redundant information available in these dual views. Furthermore, Martian terrain is "unstructured"—composed of irregular rocks, dust, and dunes—making standard pixel-to-pixel matching between left and right views notoriously difficult.

Methods

The proposed MarsSQE (Mars Stereo Quality Enhancement) framework utilizes a Bi-level Cross-view Attention network. The architecture consists of three main stages:

  1. Feature Extraction: Residual Dense Blocks (RDBs) extract initial characteristics from the compressed left and right images ($I_L, I_R$).
  2. Bi-level Cross-view Attention: This core innovation addresses the unstructured nature of Martian terrain. It employs Patch-level Attention to identify broader neighborhood contexts, followed by Pixel-level Attention for fine-grained alignment and detail recovery.
  3. Image Reconstruction: Enhanced features are fused and decoded back into high-quality stereo images ($I_L^{Enh.}, I_R^{Enh.}$).
The MarsSQE architecture featuring the bi-level cross-view attention mechanism.

The network is trained using an L1 loss function, comparing the enhanced output against original, uncompressed "raw" images.

Dataset: The Stereo Martian Image Dataset

To support this research, the authors established the first dedicated dataset for Martian stereo enhancement. It consists of 1,350 pairs of stereo images from the Perseverance rover’s Mastcam-Z, covering four primary landforms: rock, soil, sand, and sky. Analysis revealed that Martian images possess higher cross-view correlation than Earth images due to the relative uniformity of the landscape, suggesting that information lost in one view is highly likely to be recoverable from the other.

Key Results and Discussion

MarsSQE demonstrates significant improvements over both monocular and stereo baselines. Quantitatively, it achieved a Peak Signal-to-Noise Ratio (PSNR) of 31.95 dB at a Quality Factor of 30, representing a 0.19 dB improvement over the previous state-of-the-art monocular Martian enhancer (MarsQE) and 0.11 dB over the best Earth-based stereo model (iPASSR).

Qualitative comparison of MarsSQE against existing monocular and stereo enhancement methods.

Visually, MarsSQE excels at restoring sharp rock boundaries and fine soil textures where other models produce "smudged" results or artificial noise. Furthermore, the authors developed lightweight variants (MarsSQE-S and MarsSQE-M). Notably, their smallest model (1.00M parameters) outperformed much larger Earth-based models (1.37M parameters), highlighting the importance of domain-specific architecture over raw model size.

Conclusion

MarsSQE provides a robust solution for enhancing compressed Martian imagery by leveraging stereo redundancy and a bi-level attention mechanism tailored for unstructured terrain. Future work may involve integrating semantic awareness (e.g., distinguishing "rock" from "sky") to further boost accuracy or exploring on-rover deployment of lightweight versions to optimize data before transmission.

Summary updated

Abstract

Stereo images captured by Mars rovers are transmitted after lossy compression due to the limited bandwidth between Mars and Earth. Unfortunately, this process results in undesirable compression artifacts. In this paper, we present a novel stereo quality enhancement approach for Martian images, named MarsSQE. First, we establish the first dataset of stereo Martian images. Through extensive analysis of this dataset, we observe that cross-view correlations in Martian images are notably high. Leveraging this insight, we design a bi-level cross-view attention-based quality enhancement network that fully exploits these inherent cross-view correlations. Specifically, our network integrates pixel-level attention for precise matching and patch-level attention for broader contextual information. Experimental results demonstrate the effectiveness of our MarsSQE approach.
BibTeX
@misc{xu2024marssqe,
  author = {Mai Xu and Yinglin Zhu and Qunliang Xing and Jing Yang and Xin Zou},
  title = {{MarsSQE: Stereo Quality Enhancement for Martian Images Using Bi-level Cross-view Attention}},
  year = {2024},
  eprint = {2412.20685},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.20685},
}
Figure