Abstract
Blind Image Quality Assessment (IQA) is a critical task in computer vision, yet traditional deep learning models often function as "black boxes," providing numerical scores without semantic justification. This paper introduces kgMBQA, a novel framework that integrates a Quality Knowledge Graph to generate human-readable explanations alongside quality predictions. By aligning visual features with structured semantic data and leveraging Large Language Models (LLMs) for refinement, kgMBQA achieves state-of-the-art accuracy across five major benchmarks. Our approach demonstrates that semantic context not only improves interpretability but also enhances the precision of numerical quality scores.
Introduction
Predicting how humans perceive image quality without a reference image—known as Blind IQA—is challenging due to the diversity of distortions and scene complexities. Existing unimodal models often struggle with representational depth and lack transparency. A single score (e.g., 0.75) fails to communicate whether an image suffers from motion blur, noise, or poor lighting.
The kgMBQA framework addresses these limitations by moving beyond pixel-level analysis. It constructs a structured "map" of the image, linking detected entities to quality descriptors, and uses this data to generate concise, factually grounded explanations.
Methodology
The kgMBQA architecture is built upon two primary modules: Quality Knowledge Graph Construction (QKGC) and Multimodal Quality Learning (MQL).
Quality Knowledge Graph Construction (QKGC)
The QKGC module transforms raw visual data into a structured semantic representation. It first performs Entity Extraction to identify objects and their spatial coordinates. These entities are then linked via Relation Construction to quality descriptors based on distortion intensity. Finally, a Large Language Model (Llama 3.2) processes this graph data to produce a coherent, human-like explanation, ensuring the output remains grounded in the image's actual content to avoid "hallucinations."
Multimodal Quality Learning (MQL)
The MQL module processes the image and its generated textual description in parallel to arrive at a final quality score.
As shown in the architecture above, visual features are extracted using a ResNet-50 backbone, while a Transformer-based encoder handles the text. These modalities are fused using a multi-head self-attention mechanism. A key innovation is the use of contrastive learning, which aligns image features and text descriptions in a shared feature space, ensuring that the semantic "hints" directly inform the quality prediction.
Experimental Results
The model was rigorously evaluated on five diverse datasets: KonIQ-10k, LIVE Challenge, BIQ2021, TID2013, and AIGC-3K.
- State-of-the-Art Performance: kgMBQA outperformed ten leading methods, including MANIQA and Re-IQA. On the KonIQ-10k dataset, it achieved a Spearman Rank Correlation Coefficient (SRCC) of 0.9442.
- Enhanced Interpretability: Qualitative analysis confirms that the model accurately identifies specific artifacts, such as "low light noise" or "poor clarity," matching human Mean Opinion Scores (MOS) closely.
- Robustness: The multimodal approach demonstrated superior generalization in cross-dataset testing compared to traditional unimodal methods.
Discussion and Future Work
A significant finding of this research is that semantic descriptions actually improve numerical accuracy; the text provides context that helps the model disambiguate complex distortions. However, the framework introduces higher computational overhead due to the LLM and knowledge graph construction. Future work will focus on optimizing these components for real-time applications and refining spatial reasoning to better describe localized distortions within complex compositions.
Summary updated