Application of Structural Similarity Analysis of Visually Salient Areas and Hierarchical Clustering in the Screening of Similar Wireless Capsule Endoscopic Images

Rui Nie, Huan Yang, Hejuan Peng, Wenbin Luo, Weiya Fan, Jie Zhang, Jing Liao, Fang Huang, Yufeng Xiao

Abstract

Wireless Capsule Endoscopy (WCE) is a non-invasive diagnostic tool for small intestine examination, yet it produces a massive volume of redundant images (60,000–120,000 per procedure), leading to significant diagnostic fatigue and potential oversight of lesions. This paper proposes a two-stage image screening framework to reduce redundancy. The first stage utilizes agglomerative hierarchical clustering based on global color profiles. The second stage identifies visually salient sub-image blocks and applies Structural Similarity Index (SSIM) analysis to these localized regions. Experimental results on 52 clinical cases demonstrate that this method can reduce diagnostic time from 78 to 18 minutes, achieving 100% recall of abnormalities at a 66.72% reduction ratio.

Introduction

The primary challenge in WCE is the "needle in a haystack" problem. Gastroenterologists must spend 2–3 hours reviewing a single study, where the vast majority of frames depict healthy tissue. Redundancy arises from the slow movement of the capsule. While automated classification via deep learning is an active area of research, these "black box" models often lack the interpretability required for clinical trust. This study addresses the redundancy problem by developing a transparent, two-stage filtering system that prioritizes localized structural changes in salient areas.

Methodology

The proposed screening method operates in two distinct phases:

1. Agglomerative Hierarchical Clustering

Initially, images are grouped by global color characteristics. Images are converted to the HSV (Hue, Saturation, Value) color space, and a 1,000-dimensional histogram vector is generated for each. An agglomerative hierarchical clustering algorithm then constructs a distance tree to group similar frames, effectively partitioning the video into clusters of visually related content.

2. Saliency-SSIM Keyframe Extraction

Within each cluster, the system identifies "visually salient" areas—regions that contrast significantly with the background, often indicating potential lesions. Using the Lab color space and Gaussian filtering, a saliency map is produced. The algorithm then selects the top three most salient 40x40 pixel blocks.

The proposed two-stage screening pipeline, from input images to keyframe selection.

Instead of comparing entire frames, which can be dominated by static healthy tissue, the system calculates the SSIM for these specific salient blocks across adjacent frames. If the structural similarity is high, the frame is discarded as redundant. If the similarity is low, indicating a change in a potentially critical area, the frame is retained as a keyframe.

Experimental Results

The system was evaluated using a dataset of 68,400 images from 52 clinical cases, featuring 17 types of lesions. Performance was benchmarked against the manual clustering results of five independent gastroenterologists.

Identification of visually salient sub-image blocks (red squares) on a WCE frame containing a potential lesion.

Efficiency and Safety

The implementation of this screening method reduced the average review time from 78 minutes to 18 minutes. At a reduction ratio of 66.72%, the system achieved 100% recall for all abnormal categories. When the reduction ratio was increased to 79.09% (matching human performance), the recall remained high at 94.23%, with missed items being clinically minor abnormalities.

Conclusion

By combining global clustering with localized saliency-based structural analysis, this method effectively mitigates the data bottleneck in WCE. The approach preserves the clinician's role as the final diagnostic authority while significantly reducing the cognitive load and time required for review. Future work will focus on refining feature extraction to capture even the most subtle lesions at higher reduction ratios.

Summary updated

Abstract

Small intestinal capsule endoscopy is the mainstream method for inspecting small intestinal lesions, but a single small intestinal capsule endoscopy will produce 40,000- 120,000 images, the majority of which are similar and have no diagnostic value. It takes 1-3 hours for doctors to identify lesions from these images. This is time consuming and increases the possibility of misdiagnosis and missed diagnosis, because doctors may experience visual fatigue while paying attention to a large number of similar images for a long time. In order to solve these problems, we proposed a similar wireless capsule endoscope (WCE) image screening method based on structural similarity analysis and the hierarchical clustering of visually salient sub-image blocks. The similarity clustering of images was automatically identified by hierarchical clustering based on the hue, saturation, value (HSV) spatial color characteristics of the images, and the keyframe images were extracted based on the structural similarity of the visually salient sub-image blocks, in order to accurately identify and screen out similar small intestinal capsule endoscopic images. Subsequently, the proposed method was applied to the capsule endoscope imaging workstation. After screening out similar images in the complete data gathered by the Type I OMOM Small Intestinal Capsule Endoscope from 52 cases covering 17 common types of small intestinal lesions, we obtained a lesion recall of 100% and an average similar image reduction ratio of 76%. With similar images screened out, the average play time of the OMOM image workstation was 18 minutes, which greatly reduced the time spent by doctors viewing the images.
BibTeX
@misc{nie2020application,
  author = {Rui Nie and Huan Yang and Hejuan Peng and Wenbin Luo and Weiya Fan and Jie Zhang and Jing Liao and Fang Huang and Yufeng Xiao},
  title = {{Application of Structural Similarity Analysis of Visually Salient Areas and Hierarchical Clustering in the Screening of Similar Wireless Capsule Endoscopic Images}},
  year = {2020},
  eprint = {2004.02805},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.02805},
}
Figure