Exploring Cross-Modal Mutual Prompt Learning for Video Quality Assessment

Pengfei Chen, Leida Li, Jinjian Wu, Jiebin Yan, Vinit Jakhetiya, Aladine Chetouani

Abstract

Enhancing video quality assessment (VQA) through semantic information integration is a critical research focus. Recent research has employed the Contrastive Language-Image Pre-training (CLIP) model as a foundation to improve semantic perception. However, the image-text alignment inherent in these pre-trained Vision-Language (VL) models frequently results in suboptimal VQA performance. While prompt engineering has recently targeted the language component to address this alignment issue, the unique insights resided in visual analysis is still overlooked for further advancing VQA tasks. Additionally, seeking a trade-off between quality separability and domain invariance in VQA remains largely unresolved within the VL paradigm. In this paper, we introduce a novel cross-modal prompt-based approach to tackle these challenges. Specifically, we propose learnable prompts within the vision branch to foster synergy between visual and language modalities through a language-to-vision coupling function. The multi-view backbone is then carefully crafted with content enhancement and distortion-aware temporal modulation to ensure quality separability. The language prompts, derived from visual representations, are further supported by adaptive weighting mechanisms to optimize the balance between quality separability and domain invariance. Experimental results demonstrate the effectiveness of our proposed method over leading VQA models, showing significant improvements in generalization across diverse datasets. The source code for this work is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/cpf0079/CM2PL</uri>.
BibTeX
@article{chen2026exploring,
  author = {Pengfei Chen and Leida Li and Jinjian Wu and Jiebin Yan and Vinit Jakhetiya and Aladine Chetouani},
  title = {{Exploring Cross-Modal Mutual Prompt Learning for Video Quality Assessment}},
  journal = {IEEE Trans. Multimedia},
  year = {2026},
  doi = {10.1109/tmm.2025.3639890},
}

Loading related content...

Figure