CLIP Based Region-Aware Feature Fusion for Automated BBPS Scoring in Colonoscopy Images

Yujia Fu, Zhiyu Dong, Tianwen Qian, Chenye Zheng, Danian Ji, Linhai ZhuoarXiv 2025 · DOI · arXiv

Introduction

The clinical efficacy of colonoscopy is fundamentally tied to the quality of bowel preparation. Inadequate cleansing can lead to missed polyps or lesions, directly impacting patient outcomes. To standardize the assessment of bowel cleanliness, clinicians utilize the Boston Bowel Preparation Scale (BBPS), which assigns a score from 0 (unprepared) to 3 (excellent) for different segments of the colon. However, manual BBPS scoring is inherently subjective, leading to significant inter-observer variability. Furthermore, the development of automated scoring systems has been hindered by a lack of diverse, high-quality datasets.

The BBPS Scoring Challenge

Existing automated solutions often struggle with the "messy" reality of clinical practice because they are trained on small, homogeneous datasets. For example, the widely used NERTHU dataset contains images from only 10 subjects. To address these limitations, researchers have developed a new framework that leverages large-scale pre-trained models and a more robust dataset.

Figure 1: Overview of the BBPS scoring system, example images for each score (0-3), and the expert consensus process used to create the HDFD dataset.

Methodology: Dual-Branch Feature Fusion

The proposed Region-Aware Feature Fusion Network utilizes a dual-branch architecture built upon the CLIP (Contrastive Language-Image Pre-training) foundation model. This approach combines global context with localized feature detection:

  • Visual Branch (Global): Employs a CLIP ViT-B/16 backbone with a lightweight adapter module. This allows the model to adapt to medical imagery while retaining the broad feature-extraction capabilities of the original CLIP model.
  • Fecal Branch (Local): Because BBPS scores are heavily influenced by the presence and type of stool, this branch focuses on localized regions. It uses 180 multi-scale "anchors" to zoom in on potential fecal matter. Instead of complex segmentation, it utilizes textual priors (e.g., "yellow stool") to align image regions with descriptive text via CLIP’s joint embedding space.
  • Gated Fusion: A gated-attention mechanism dynamically integrates global and local features, ensuring that specific fecal cues are prioritized only when they are relevant to the overall score.

Data Diversity and Visualization

A key contribution of this work is the introduction of the High-quality Diverse Fecal Dataset (HDFD). Containing 2,240 images from 517 subjects, HDFD provides a much more realistic representation of clinical variety than previous benchmarks. To ensure ground-truth reliability, each image was only included if three independent experts reached a unanimous consensus on the score.

Figure 2: t-SNE visualizations of image features. Plot (a) shows the HDFD dataset, where overlapping clusters reflect the clinical ambiguity between scores. Plot (b) shows the NERTHU dataset, where distinct clusters indicate a lack of real-world diversity.

As shown in the t-SNE visualizations, the HDFD dataset captures the "fuzzy boundaries" between scores 1 and 2 that clinicians frequently encounter. In contrast, the NERTHU dataset presents overly distinct clusters, which can lead to inflated performance metrics that do not generalize to real-world settings.

Key Results and Performance

The proposed model achieved a state-of-the-art 91.00% accuracy on the HDFD dataset, significantly outperforming standard CLIP baselines. On the smaller NERTHU dataset, the model achieved 100% accuracy, demonstrating its superior generalization capabilities. Ablation studies confirmed that the inclusion of the local "Fecal-feature" branch was critical, contributing a 2.75% boost in overall accuracy.

Conclusion and Future Directions

By combining global image context with region-aware fecal detection, this framework moves automated BBPS scoring closer to clinical utility. Future research will focus on adapting this frame-based approach for real-time video processing during live colonoscopies and expanding textual priors to include other distractors like bubbles or watery fluids.

Summary updated

Abstract

Accurate assessment of bowel cleanliness is essential for effective colonoscopy procedures. The Boston Bowel Preparation Scale (BBPS) offers a standardized scoring system but suffers from subjectivity and inter-observer variability when performed manually. In this paper, to support robust training and evaluation, we construct a high-quality colonoscopy dataset comprising 2,240 images from 517 subjects, annotated with expert-agreed BBPS scores. We propose a novel automated BBPS scoring framework that leverages the CLIP model with adapter-based transfer learning and a dedicated fecal-feature extraction branch. Our method fuses global visual features with stool-related textual priors to improve the accuracy of bowel cleanliness evaluation without requiring explicit segmentation. Extensive experiments on both our dataset and the public NERTHU dataset demonstrate the superiority of our approach over existing baselines, highlighting its potential for clinical deployment in computer-aided colonoscopy analysis.
BibTeX
@misc{fu2025clip,
  author = {Yujia Fu and Zhiyu Dong and Tianwen Qian and Chenye Zheng and Danian Ji and Linhai Zhuo},
  title = {{CLIP Based Region-Aware Feature Fusion for Automated BBPS Scoring in Colonoscopy Images}},
  year = {2025},
  eprint = {2512.20374},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.20374},
}
Figure