MarsQE: Semantic-Informed Quality Enhancement for Compressed Martian Image

C. Liu, Mai Xu, Qunliang Xing, Xin ZouIEEE TCSVT 2025 · DOI

TL;DR

This paper introduces MarsQE, a novel semantic-informed approach to enhance the quality of compressed Martian images. By leveraging unique characteristics of Martian imagery, such as high inter- and intra-image similarity and compact texture representations, MarsQE significantly outperforms existing methods, paving the way for more accurate geological analysis and scientific discovery from Mars exploration.

What Problem Does This Paper Address?

Mars exploration missions rely on transmitting vast amounts of image data back to Earth. Due to limited bandwidth, lossy compression techniques like JPEG are essential. However, this compression introduces visual artifacts (e.g., ringing, blocking) that can obscure crucial details, hindering geological analysis of the Martian surface. Existing image enhancement methods, primarily developed for Earth imagery, fail to adequately address the unique semantic properties of Martian images, leading to suboptimal results. This paper aims to bridge this gap by developing a specialized quality enhancement technique tailored for Martian imagery.

Methods

MarsQE employs a two-phase approach:

  1. Semantic-based Reference Matching (SMM): This module leverages the observed high inter-image similarity in Martian data. It first categorizes Martian image patches into semantic classes (e.g., sky, sand, soil, rock) based on existing segmentation methods. Then, for each compressed patch, it searches for semantically similar patches within a pre-constructed reference dictionary. This allows the system to find relevant texture patterns from other Martian images.
  2. Semantic-informed Quality Enhancement (QEM): This phase utilizes the matched reference patches to guide the enhancement of the compressed input. It employs a L-level encoder-decoder architecture. Texture Extracting Modules (TEMs) extract semantic-related features from compressed, raw reference, and compressed reference patches. These features are then fused by Texture Fusing Modules (TFMs) to enrich the decoded features, aiming to restore texture details.

A post-enhancement network is also included to further mitigate blocking artifacts and refine textures, particularly important for patch-wise enhancement methods. This network incorporates a Non-Local Module (NLM) and Quality Enhancement Modules (QEMs) for improved detail restoration.

Datasets

  • MIC (Martian Image Compression) dataset: This dataset, consisting of high-resolution Martian color images captured by the Perseverance rover's Mastcam-Z camera, was used for training, validation, and testing. It contains 3,088 training, 386 validation, and 386 testing images.
  • AI4MARS dataset: A large-scale labeled dataset with over 35K images from NASA's Curiosity, Opportunity, and Spirit rovers, used for qualitative evaluation.
  • Mars32K dataset: Contains 32,368 color Martian images collected by the Curiosity rover, also used for qualitative evaluation.

The paper notes that AI4MARS and Mars32K lack ground-truth images, making them suitable only for qualitative assessments.

Key Insights and Ideas

  • Martian images are surprisingly similar: Unlike the diverse Earth imagery, Martian images exhibit significantly higher inter-image and intra-image similarity. This means that patches from different Martian images, or even within the same image, often share similar visual characteristics. This insight is crucial for leveraging reference-based enhancement.
  • Martian textures are "compact": Martian images can be effectively categorized into a limited number of distinct semantic classes (e.g., sky, sand, soil, rock), and these classes have unique texture patterns. This contrasts with Earth images, which have a much wider variety of textures and semantics. This "compactness" allows for more targeted and effective reference matching.
  • Semantics are key for Martian enhancement: Standard enhancement techniques designed for Earth images fail because they don't account for the specific semantic properties of Martian terrain. MarsQE's success hinges on explicitly using these semantic categories to guide the enhancement process.

Key Results

  • Quantitative Performance: MarsQE achieved state-of-the-art results on the MIC test set, outperforming established methods like AR-CNN, DnCNN, RBQE, and DAQE across various metrics (PSNR, PSNR-B, SSIM, MS-SSIM, LPIPS, MUSIQ). For JPEG compression with QF=30, MarsQE achieved a PSNR of 34.61 dB, which was 2.17 dB higher than the baseline and 0.12 dB higher than the second-best approach.
  • Rate-Distortion Performance: MarsQE demonstrated superior rate-distortion performance, achieving significant BD-rate reductions (e.g., 38.94% for PSNR with JPEG baseline).
  • Qualitative Performance: Visual comparisons showed MarsQE effectively restoring details, suppressing compression artifacts (like ringing and blocking), and producing clearer images, especially in challenging areas like rock edges and skies.
  • Efficiency: A lighter version, MarsQE-lite, achieved competitive performance with significantly fewer parameters and FLOPs compared to other methods, demonstrating its potential for resource-constrained environments.
  • Generalization: The MarsQE model showed good generalization capabilities, performing well on datasets from different Martian missions (AI4MARS, Mars32K) without fine-tuning, even with unknown compression parameters and grayscale images.

Limitations and Future Work

  • Scene Limitation: While MarsQE has good generalization, it can struggle with highly complex cases or entirely new terrain types not represented in its reference dictionary.
  • Task Separation: Currently, terrain segmentation and quality enhancement are treated as separate tasks. The authors suggest that these could be mutually beneficial.
  • Future Directions: The authors propose extending MarsQE by integrating a dynamic reference dictionary and jointly training segmentation and enhancement tasks to further improve generalization and overall performance.

Summary updated

Abstract

Lossy image compression is essential for Mars exploration missions, due to the limited bandwidth between Earth and Mars. However, the compression may introduce visual artifacts that complicate the geological analysis of the Martian surface. Existing quality enhancement approaches, primarily designed for Earth images, fall short for Martian images due to a lack of consideration for the unique Martian semantics. In response to this challenge, we conduct an in-depth analysis of Martian images, yielding two key insights based on semantics: the presence of texture similarities and the compact nature of texture representations in Martian images. Inspired by these findings, we introduce MarsQE, an innovative, semantic-informed, two-phase quality enhancement approach specifically designed for Martian images. The first phase involves the semantic-based matching of texture-similar reference images, and the second phase enhances image quality by transferring texture patterns from these reference images to the compressed image. We also develop a post-enhancement network to further reduce compression artifacts and achieve superior compression quality. Our extensive experiments demonstrate that MarsQE significantly outperforms existing approaches for Earth images, establishing a new benchmark for the quality enhancement on Martian images. The code is available at https://github.com/keriphLiu/MarsQE.
BibTeX
@article{liu2025marsqe,
  author = {C. Liu and Mai Xu and Qunliang Xing and Xin Zou},
  title = {{MarsQE: Semantic-Informed Quality Enhancement for Compressed Martian Image}},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  doi = {10.1109/tcsvt.2025.3643953},
}

Loading related content...

Figure