TL;DR
This paper introduces a novel deep learning approach for no-reference image quality assessment (NR-IQA) specifically tailored for adaptive optics (AO) images. By training a neural network to predict image quality based on the similarity between the image's point spread function (PSF) and an ideal Airy spot, the method achieves high accuracy and outperforms traditional IQA techniques on both simulated and real AO data.
What Problem Does This Paper Address?
Adaptive optics (AO) systems, crucial for improving image quality from ground-based telescopes by correcting atmospheric turbulence, still produce images with residual degradations. These degradations, stemming from factors like atmospheric turbulence, tracking errors, and sensor noise, make image quality assessment (IQA) challenging. Current methods, relying on traditional IQA or manual inspection, are often unreliable and inefficient. Existing NR-IQA methods, designed for general image quality, struggle to adapt to the unique characteristics of AO images, such as their grayscale nature, presence of motion blur and atmospheric halos, and large background regions with little semantic information. This paper aims to bridge this gap by developing a robust and accurate NR-IQA method specifically for extended target AO images.
Methods
The core of this work is a supervised deep neural network designed for NR-IQA of AO images. Instead of using image features directly as labels, the proposed method leverages the Point Spread Function (PSF) of the degraded image. The supervised signal is derived from the normalized correlation coefficient between the image's PSF and an ideal Airy spot, which represents the theoretical diffraction pattern of a point source in an aberration-free optical system. A higher correlation indicates better image quality.
The network architecture consists of three main modules:
- Input Adaptive Module: This module preprocesses the input image (512x512, grayscale) using operations like
SpaceToDepthandDICEUnitto reduce downsampling-induced information loss and fuse features across channels. It also includes a residual block. - Multi-Scale Feature Extraction Module: This module employs a ResNet50 backbone to extract features at various scales. It concatenates feature maps from different stages of ResNet50 to capture a comprehensive representation of image degradation.
- Quality Prediction Network: A simple network with a fully connected layer and a sigmoid activation function maps the extracted multi-scale features to a final quality score. The mean squared error is used as the loss function.
Datasets
The authors created a novel dataset of 400,000 simulated AO extended target images. These images were generated by:
- Rendering 3D models of extended objects under varying lighting, pose, and distance.
- Degrading these rendered images with simulated atmospheric turbulence (modeled using Kolmogorov spectrum with RMS values between 0.1 and 0.5) and motion blur (lengths between 3 and 9 pixels, with random directions).
- Assigning quality labels based on the normalized correlation coefficient between the simulated PSF and the ideal Airy spot.
The dataset was split randomly into training, validation, and testing sets in a 4:1:1 ratio. The method was also validated using actual acquired AO images from a laboratory-built AO platform.
Key Insights and Ideas
- PSF as a Superior Quality Metric: The paper strongly advocates for using the PSF's similarity to an ideal Airy spot as a more effective quality label for AO images than traditional image features. This approach directly captures the impact of degradation factors on the optical system's response.
- Domain-Specific IQA is Crucial: General-purpose IQA methods struggle with the unique characteristics of AO images. The proposed method's success highlights the need for domain-specific solutions.
- Deep Learning for PSF-Image Relationship: The deep neural network effectively learns the complex, non-linear relationship between image degradation (as reflected in the PSF) and perceived image quality, even when the PSF itself is unknown during real-time acquisition.
- Simulated Data Quality Matters: The careful construction of the simulated dataset, incorporating realistic degradation models and generating labels based on physical principles (PSF vs. Airy spot), was critical for training a high-performing model.
Key Results
- High Correlation on Simulated Data: The proposed method achieved a Spearman's rank correlation coefficient (SRCC) of 0.97 on the simulated test data.
- Outperforms Baselines: On both simulated and real AO images, the proposed method significantly outperformed traditional methods like Tenengrad and LOG, as well as the general-purpose deep learning method HyperIQA. Table 1 shows SRCC values of 0.971 for the proposed method compared to 0.673 (Tenengrad), 0.769 (LOG), and 0.656 (HyperIQA) on real images.
- Visual Agreement: Figure 13 demonstrates that the proposed method's predicted scores closely align with the actual degree of AO system correction, a correlation that is less precise for the other methods.
- Efficient Performance: The IQA speed is approximately 61 FPS, meeting online IQA requirements for AO extended target images.
Limitations and Future Work
- Unknown PSF in Real-World Scenarios: The paper acknowledges that the PSF is often unknown during actual image acquisition, necessitating the use of deep networks to infer the relationship between image features and PSF-based quality scores.
- Future Work: The authors plan to improve the network structure and expand the dataset. They also aim to extend the method's application to assessing image quality affected by atmospheric turbulence more broadly.
Summary updated