TL;DR
This paper introduces FreqAlign, a novel approach to improve blind image quality assessment (BIQA) models' ability to generalize to new, unseen image datasets. By analyzing image features in the frequency domain, FreqAlign identifies and leverages the most transferable perceptual information, significantly outperforming existing methods in cross-domain BIQA tasks.
What Problem Does This Paper Address?
Blind Image Quality Assessment (BIQA) models aim to predict image quality without a reference image. A major challenge is that models trained on one type of image degradation (e.g., synthetic distortions) often perform poorly on another (e.g., authentic distortions) due to distribution shifts. Existing Unsupervised Domain Adaptation (UDA) techniques for BIQA typically align features in the low-frequency domain, which is insufficient as crucial perceptual information can be lost. This paper addresses the need to effectively transfer perceptual knowledge across different image domains by exploring the entire frequency spectrum of image features.
Methods
FreqAlign tackles the UDA problem in BIQA by focusing on the frequency domain of deep features. The core idea is to decompose image features into their frequency components using the Discrete Cosine Transform (DCT). Instead of aligning all frequencies, FreqAlign employs a "frequency movement" strategy. This involves:
- Frequency Decomposition: Extracting features from a deep neural network and decomposing them into an 8x8 frequency matrix for each channel.
- Frequency Movement-based Alignment: This is the key innovation. It uses a sliding window to iteratively select and align specific frequency bands. This process is guided by a transferability metric (e.g., MMD) to identify bands that best transfer perceptual knowledge.
- Warm-up and Perturbation: The process includes a warm-up phase to stabilize the model and a perturbation-based finetuning step to enhance robustness and tolerance to sub-optimal frequency selections.
- Adversarial Alignment: Within the selected frequency bands, adversarial alignment is used to make the source and target domain features indistinguishable.
- Overall Optimization: The model is optimized using a combination of adversarial loss, regression loss, and a self-supervised loss for target domain data.
Datasets
The paper evaluates FreqAlign on several standard BIQA datasets:
- Synthetic Distortion Datasets: KADID10k, CSIQ
- Authentic Distortion Datasets: KonIQ-10k, CLIVE, BID (realistic blur dataset)
These datasets are used to construct various cross-domain scenarios, including synthetic-to-authentic, authentic-to-synthetic, and cross-distortion settings.
Key Insights and Ideas
- Perceptual Knowledge is Multi-Frequency: The most significant insight is that transferable perceptual knowledge for BIQA is not confined to low frequencies but is distributed across various frequency components.
- Frequency-Specific Transferability: The optimal frequency bands for transferability vary depending on the specific source and target domains, highlighting the need for adaptive selection.
- Frequency Movement for Efficient Selection: A sliding window approach to "move" through frequency bands allows for efficient identification of optimal bands without exhaustively testing every single frequency.
- Warm-up and Perturbation Enhance Stability: The proposed warm-up and perturbation strategies are crucial for achieving stable frequency selection and robust alignment.
Key Results
FreqAlign demonstrates significant improvements in cross-domain BIQA performance across various settings:
- Synthetic to Authentic: FreqAlign achieves state-of-the-art results, outperforming existing methods by a substantial margin (e.g., an average gain of 0.0791/0.1099 on SROCC/PLCC compared to DANN).
- Authentic to Synthetic: FreqAlign also shows strong performance, outperforming the baseline by an average of 0.0878/0.0870 on SROCC/PLCC.
- Cross-Distortion Types: FreqAlign exhibits stable and remarkable transferability across all seven distortion types in KADID10k, outperforming the second-best UDA method by a significant margin.
- Aesthetic Quality Assessment: As an extension, FreqAlign also shows effectiveness in aesthetic quality assessment, demonstrating its generalizability.
Limitations and Future Work
- Trajectory Dependence: The paper notes that different frequency movement trajectories (left-to-right, up-to-down, zig-zag) yield varying results, suggesting that the optimal trajectory might depend on the specific frequency distribution of transferability. The "left-to-right" trajectory was found to be most effective in their experiments.
- Move Interval Length: The optimal length of the move interval (number of epochs) for frequency movement was found to be dataset-dependent, requiring tuning.
- Number of Frequencies in Window: The optimal number of frequencies within the sliding window was explored, with a peak performance observed around 10 frequencies.
- Transferability Metrics: While MMD was found to be optimal, CORAL and adversarial loss also showed comparable performance, suggesting flexibility in metric choice.
- Future Work: The paper implicitly suggests further exploration into optimal trajectory selection and potentially more sophisticated methods for identifying complementary frequency bands. The success in aesthetic quality assessment also opens avenues for applying FreqAlign to other related vision tasks.
Summary updated