Abstract
Radio astronomy studies the Universe by observing the radio emissions of celestial bodies. Different methods can be used to recover the sky brightness distribution (SBD), which describes the distribution of celestial sources from recorded data, with the output dependent on the method used. Image quality assessment (IQA) indexes can be used to compare the differences between restored SBDs produced by different image reconstruction techniques to evaluate the effectiveness of different techniques. However, reconstructed images (for the same SBD) can appear to be very similar, especially when observed by the human visual system (HVS). Hence current structural similarity methods, inspired by the HVS, are not effective. In the past, we have proposed two methods to assess point source images, where low amounts of concentrated information are present in larger regions of noise-like data. But for images that include extended source(s), the increase in complexity of the structure makes the IQA methods for point sources over-sensitive since the important objects cannot be described by isolated point sources. Therefore, in this article we propose augmented Low-Information Similarity Index (augLISI), an improved version of LISI, to assess images including extended source(s). Experiments have been carried out to illustrate how this new IQA method can help with the development and study of astronomical imaging techniques.
The Challenge of Low-Information Images
In radio astronomy, images are often "low-information"—meaning a few bright celestial sources are scattered across a vast, noise-filled background. When researchers develop new image reconstruction algorithms (like CLEAN or MEM), they need a way to mathematically prove that one version is better than another.
Standard metrics like SSIM (Structural Similarity Index) were built to mimic the Human Visual System (HVS), which is highly sensitive to phase and edges. In the sparse, noisy world of radio astronomy, SSIM often performs poorly because it over-emphasizes differences in the noise-like background rather than the actual celestial sources.
While the previously developed LISI worked for point sources (stars), it was found to be "over-sensitive" when dealing with extended sources (like nebulae or galaxies). Because extended sources have complex internal structures, LISI would often report a low similarity score even if the primary scientific features were well-recovered, simply because it was distracted by low-intensity variations.
Introducing augLISI
To overcome these hurdles, we propose augLISI (augmented Low-Information Similarity Index). This metric focuses on high-intensity source data while remaining robust against the noise-heavy backgrounds typical of interferometric imaging.
The key innovations include:
- Intensity Weighting: The formula ensures the metric is less sensitive to differences in low-intensity (noise) regions and more focused on high-intensity (source) regions.
- Global Normalization: The denominator is the sum of all pixels in both images, preventing a single bright image from unfairly dominating the comparison.
- Joint Analysis (Tiling): The images are divided into regular tiles to calculate both SSIM and augLISI. By comparing these two values, researchers can categorize differences into scenarios, such as identifying where bright structures are similar but faint structures differ.
Key Insights and Results
The performance of augLISI was evaluated using both simulated and real-world data, including the supernova remnant SNR G55.7+3.4 and the binary black hole system 3C 75.
Stability in Noise
augLISI is remarkably more stable than SSIM when noise levels increase. While SSIM scores drop sharply as noise is added (even if the source remains identical), augLISI remains near 1.0, correctly identifying that the primary scientific information hasn't changed. In simulations where noise was increased from 0% to 4.42%, the standard deviation of augLISI remained significantly lower (0.0002–0.0004) compared to SSIM (up to 0.0031).
Algorithm Benchmarking
The metric successfully quantified that IUWT (a compressed sensing method) and MS-CLEAN were more similar to each other in recovering high-intensity structures than they were to MEM (Maximum Entropy Method). Furthermore, when comparing MS-CLEAN results from different software packages (CASA vs. WSClean), augLISI yielded scores above 0.99, proving that different implementations produce nearly identical scientific results despite different noise patterns.
Conclusion
augLISI provides a reliable tool for assessing astronomical image quality without the need for complex "source extraction" (segmenting stars/galaxies first). By mathematically weighting sources higher, it saves computational effort and avoids errors in segmentation. Future directions include integrating augLISI into automated software development pipelines and expanding its use to transient search applications.
Summary updated