MMAG: Multimodal Learning for Mucus Anomaly Grading in Nasal Endoscopy via Semantic Attribute Prompting

Xinpan Yuan, Mingzhu Huang, Liujie Hua, Jianuo Ju, Xu ZhangEMNLP 2025

TL;DR

This paper introduces MMAG, a novel multimodal learning framework for accurately grading nasal mucus anomalies in endoscopic images. By combining attribute-guided localization with rank-aware severity assessment, MMAG achieves significant improvements in detection and grading, offering a more interpretable and clinically relevant solution for rhinitis assessment.

What Problem Does This Paper Address?

Accurate grading of rhinitis severity relies heavily on analyzing mucus secretions in nasal endoscopy images. However, these secretions, particularly clear nasal discharge (CND) and purulent nasal secretion (PUS), are often ambiguous and structurally variable, making automated grading challenging, especially under weak supervision. Existing methods struggle with:

  • Ambiguity and Variability: Mucus can be transparent, reflective, and conform to complex anatomical surfaces, making it hard to segment or classify using traditional methods.
  • Ordinal Nature of Severity: Rhinitis severity progresses along a continuous spectrum, but many methods treat it as a fixed-class classification problem, ignoring this ordinal relationship and limiting interpretability.
  • Weak Supervision: Obtaining large, precisely annotated datasets for medical imaging is difficult, necessitating methods that can learn effectively from limited or noisy labels.

This paper aims to bridge this gap by developing a framework that can precisely locate mucus anomalies and grade their severity in an interpretable, ordinal manner, even with limited annotations.

Methods

The proposed MMAG framework comprises three key components:

  1. Attribute Prompt Construction (APC): Clinical attributes (e.g., secretion type, color, location) are converted into natural language prompts (e.g., "the secretion type is PUS") using a template like "a photo of the [Attribute]". These prompts guide the alignment between visual and textual information.
  2. Anomaly-Aware Region Alignment (AARA): This module uses a pre-trained CLIP visual encoder with lightweight adapters to extract multi-level image features. These features are then aligned with the attribute prompts via a cross-modal projector. A dual-branch design supports both zero-shot (no labeled data) and few-shot (limited labeled data) inference for anomaly classification and localization. This approach enhances robustness in low-contrast and complex regions.
  3. Rank-Aware Severity Assessment (RSA): This component models the ordinal nature of severity. It aligns localized anomaly features with severity-ordered prompts (e.g., "no", "mild", "moderate", "severe") in a rank-aware feature space. A novel rank loss function (Lrank) is introduced to enforce correct ordering of similarity scores between image features and severity prompts, ensuring fine-grained and interpretable grading.

Datasets

The study utilizes two clinically annotated datasets for training and evaluation:

  • CND (Clear Nasal Discharge): Focuses on clear mucus secretions.
  • PUS (Purulent Nasal Secretion): Focuses on purulent mucus secretions.

These datasets are representative of the challenges in nasal endoscopy due to complex anatomy and diverse mucus presentations. The paper also mentions transfer experiments on colonoscopy datasets (Hyper-Kvasir, CVC-ColonDB, CVC-ClinicDB) to demonstrate the model's generalization ability.

Key Insights and Ideas

  • Attribute-Guided Localization: Using structured clinical attributes as prompts allows the model to focus on relevant regions for anomaly detection, improving precision, especially in challenging low-contrast scenarios.
  • Ordinal Severity Modeling: Explicitly modeling the ordered nature of rhinitis severity with rank-aware prompts and loss functions is crucial for clinically meaningful grading, moving beyond simple classification.
  • Dual-Branch (Zero-Shot/Few-Shot) Architecture: Combining zero-shot and few-shot learning paths makes the framework adaptable to varying levels of data availability, enhancing its practical utility.
  • Interpretability through Prompts: The use of natural language prompts and the resulting similarity scores provide a degree of interpretability, allowing clinicians to understand why a certain grade was assigned.

Key Results

The MMAG framework demonstrates significant performance improvements:

  • CND Dataset: ACC improved from 65.96% to 71.97%, and F1 score from 56.93% to 69.78%.
  • PUS Dataset: ACC and F1 scores reached 84.13% and 85.93%, respectively.
  • Comparison to Baselines: MMAG consistently outperformed several baselines, including GDRNet, CLIP, OrdinalCLIP, and CLIP-DR, showing better robustness under boundary ambiguity and improved specificity.
  • Qualitative Results: Visualizations show more accurate and concentrated anomaly heatmaps, with improved robustness under challenging conditions like blurred boundaries and specular reflections. The rank-aware model exhibits clearer inter-class boundaries in similarity matrices compared to direct grading baselines.
  • Transfer Learning: The framework showed strong performance when transferred to colonoscopy datasets, indicating good generalization capabilities.

Limitations and Future Work

  • Low-Illumination Challenges: The framework faces challenges in extremely low-illumination regions where mucus might be overlooked or misclassified, potentially leading to underestimation of severity.
  • Future Directions: The authors suggest exploring adaptive enhancement or uncertainty modeling techniques to improve robustness in underexposed areas. Future work will also focus on extracting features from borderline mucus and developing fine-grained recognition algorithms for secretions in hidden nasal areas to further enhance clinical discrimination.

Summary updated

BibTeX
@inproceedings{yuan2025mmag,
  author = {Xinpan Yuan and Mingzhu Huang and Liujie Hua and Jianuo Ju and Xu Zhang},
  title = {{MMAG: Multimodal Learning for Mucus Anomaly Grading in Nasal Endoscopy via Semantic Attribute Prompting}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure