Intensity-sensitive Quality Assessment of Extended Sources in Astronomical Images

Xiaotong Li, Karel Adámek, Wesley ArmourThe Astrophysical Journal Supplement Series 2024

Abstract

Radio astronomy studies the Universe by observing the radio emissions of celestial bodies. Different methods can be used to recover the sky brightness distribution (SBD), which describes the distribution of celestial sources from recorded data, with the output dependent on the method used. Image quality assessment (IQA) indexes can be used to compare the differences between restored SBDs produced by different image reconstruction techniques to evaluate the effectiveness of different techniques. However, reconstructed images (for the same SBD) can appear to be very similar, especially when observed by the human visual system (HVS). Hence current structural similarity methods, inspired by the HVS, are not effective. In the past, we have proposed two methods to assess point source images, where low amounts of concentrated information are present in larger regions of noise-like data. But for images that include extended source(s), the increase in complexity of the structure makes the IQA methods for point sources over-sensitive since the important objects cannot be described by isolated point sources. Therefore, in this article we propose augmented Low-Information Similarity Index (augLISI), an improved version of LISI, to assess images including extended source(s). Experiments have been carried out to illustrate how this new IQA method can help with the development and study of astronomical imaging techniques.

The Challenge of Low-Information Images

In radio astronomy, images are often "low-information"—meaning a few bright celestial sources are scattered across a vast, noise-filled background. When researchers develop new image reconstruction algorithms (like CLEAN or MEM), they need a way to mathematically prove that one version is better than another.

Standard metrics like SSIM (Structural Similarity Index) were built to mimic the Human Visual System (HVS), which is highly sensitive to phase and edges. In the sparse, noisy world of radio astronomy, SSIM often performs poorly because it over-emphasizes differences in the noise-like background rather than the actual celestial sources.

While the previously developed LISI worked for point sources (stars), it was found to be "over-sensitive" when dealing with extended sources (like nebulae or galaxies). Because extended sources have complex internal structures, LISI would often report a low similarity score even if the primary scientific features were well-recovered, simply because it was distracted by low-intensity variations.

Introducing augLISI

To overcome these hurdles, we propose augLISI (augmented Low-Information Similarity Index). This metric focuses on high-intensity source data while remaining robust against the noise-heavy backgrounds typical of interferometric imaging.

Comparison of LISI (top row) and augLISI (bottom row) sensitivity. The top row (a-c) shows the sharp, over-sensitive response of LISI, while the bottom row (d-f) demonstrates the smoother, more robust response of augLISI for extended sources.

The key innovations include:

  • Intensity Weighting: The formula ensures the metric is less sensitive to differences in low-intensity (noise) regions and more focused on high-intensity (source) regions.
  • Global Normalization: The denominator is the sum of all pixels in both images, preventing a single bright image from unfairly dominating the comparison.
  • Joint Analysis (Tiling): The images are divided into regular tiles to calculate both SSIM and augLISI. By comparing these two values, researchers can categorize differences into scenarios, such as identifying where bright structures are similar but faint structures differ.

Key Insights and Results

The performance of augLISI was evaluated using both simulated and real-world data, including the supernova remnant SNR G55.7+3.4 and the binary black hole system 3C 75.

Stability in Noise

augLISI is remarkably more stable than SSIM when noise levels increase. While SSIM scores drop sharply as noise is added (even if the source remains identical), augLISI remains near 1.0, correctly identifying that the primary scientific information hasn't changed. In simulations where noise was increased from 0% to 4.42%, the standard deviation of augLISI remained significantly lower (0.0002–0.0004) compared to SSIM (up to 0.0031).

2D heatmap of the augLISI response, illustrating the smoother intensity-sensitive transitions compared to the original LISI metric.

Algorithm Benchmarking

The metric successfully quantified that IUWT (a compressed sensing method) and MS-CLEAN were more similar to each other in recovering high-intensity structures than they were to MEM (Maximum Entropy Method). Furthermore, when comparing MS-CLEAN results from different software packages (CASA vs. WSClean), augLISI yielded scores above 0.99, proving that different implementations produce nearly identical scientific results despite different noise patterns.

Conclusion

augLISI provides a reliable tool for assessing astronomical image quality without the need for complex "source extraction" (segmenting stars/galaxies first). By mathematically weighting sources higher, it saves computational effort and avoids errors in segmentation. Future directions include integrating augLISI into automated software development pipelines and expanding its use to transient search applications.

Summary updated

Abstract

Abstract Radio astronomy studies the Universe by observing the radio emissions of celestial bodies. Different methods can be used to recover the sky brightness distribution (SBD), which describes the distribution of celestial sources from recorded data, with the output dependent on the method used. Image quality assessment (IQA) indexes can be used to compare the differences between restored SBDs produced by different image reconstruction techniques to evaluate their effectiveness. However, reconstructed images (for the same SBD) can appear to be very similar, especially when observed by the human visual system (HVS). Hence, current structural similarity methods, inspired by the HVS, are not effective. In the past, we have proposed two methods to assess point-source images, where low amounts of concentrated information are present in larger regions of noise-like data. But for images that include extended source(s), the increase in complexity of the structure makes the IQA methods for point sources oversensitive because the important objects cannot be described by isolated point sources. Therefore, in this article we propose the augmented low-information similarity index (augLISI), an improved version of LISI, to assess images including extended source(s). Experiments have been carried out to illustrate how this new IQA method can help with the development and study of astronomical imaging techniques. Note that although we focus on radio astronomical images herein, these IQA methods are also applicable to other astronomical images and imaging techniques.
BibTeX
@article{li2024intensity,
  author = {Xiaotong Li and Karel Adámek and Wesley Armour},
  title = {{Intensity-sensitive Quality Assessment of Extended Sources in Astronomical Images}},
  journal = {The Astrophysical Journal Supplement Series},
  year = {2024},
}
Figure