Integration of Global and Local Features for Specular Reflection Inpainting in Colposcopic Images

Xiaoxia Wang, Ping Li, Yuchun Lv, Huifeng Xue, Tianxiang Xu, Yongzhao Du, Peizhong LiuJournal of Healthcare Engineering 2021 · DOI

TL;DR

This paper introduces a novel method for removing specular reflections (SR) from colposcopic images, crucial for accurate cervical cancer diagnosis. By integrating global image smoothing with local texture preservation, their approach significantly improves visual clarity and has been shown to boost physician diagnostic accuracy.

What Problem Does This Paper Address?

Cervical cancer screening relies heavily on colposcopy, a procedure that uses a magnified view of the cervix. However, the bright light used during colposcopy can cause specular reflections (SR) on the cervical surface. These SR regions appear as bright spots that can be easily mistaken for actual lesions (like acetic white regions) or obscure important anatomical details, hindering accurate diagnosis by both automated systems and human physicians. Existing methods for removing SR often either over-smooth the image, losing crucial texture information, or fail to completely eliminate the reflections, leaving them to interfere with diagnosis. This paper aims to bridge this gap by developing an inpainting technique that effectively removes SR while preserving the fine details essential for clinical interpretation.

Methods

The proposed method employs a two-stage approach:

  • Global Processing: This stage uses a modified version of Arnold's method, originally developed for colonoscopy images. It applies a Gaussian blur to the entire image to achieve a general smoothing effect and reduce the intensity of SR regions. This step aims for overall image cleanliness.
  • Local Processing: To counteract the over-smoothing from the global stage and preserve fine textures, this stage utilizes an exemplar-based inpainting method (similar to Criminisi's approach). The image is divided into blocks, and within each block containing SR, the algorithm intelligently fills in the missing information by copying texture and structure from similar, non-SR regions within the same block. This ensures that local details are meticulously reconstructed.
  • Integration: Finally, the results from the global and local processing stages are merged. The authors convert the images to the HSV color space and combine the Hue, Saturation, and Value components, prioritizing the best values from each stage to create a final image that is both smooth and rich in detail.

Datasets

The paper uses a dataset of 300 colposcopic images from Fujian Provincial Maternity and Children's Hospital, classified into Normal, CIN1, CIN2, CIN3, and Cancer categories (65, 65, 60, 60, and 50 images respectively). For comparative evaluation, they also created a "Ref_set" by extracting 50 images from existing literature that dealt with SR removal in various endoscopic images (including colonoscopic and laparoscopic).

Key Insights and Ideas

  • Global-Local Synergy: The core insight is that a purely global or purely local approach to SR inpainting is insufficient. Combining a broad smoothing effect with meticulous local detail reconstruction is key to achieving both effective SR removal and preservation of diagnostic information.
  • HSV Color Space for Integration: Utilizing the HSV color space for integrating the global and local results is a clever way to leverage how humans perceive color and brightness, leading to a more perceptually pleasing and diagnostically useful final image.
  • Importance of Texture Preservation: The paper strongly emphasizes that for colposcopic images, preserving fine textures is paramount. Over-smoothing, even if it removes SR, can be detrimental to diagnosis.

Key Results

  • Subjective Visual Assessment: In a user study involving two groups of testers (medical anatomy and computer image processing experts), the proposed method achieved the highest average score (3.55) among five comparison sets, indicating superior visual quality.
  • Clinical Evaluation: When tested with 6 colposcopy physicians, the proposed method improved diagnostic accuracy. For a two-category classification (Normal & Lesion), accuracy increased by an average of 1.44%, and for a four-category classification (Normal & LSIL & HSIL & Cancer), accuracy increased by an average of 2.03%.
  • Parameter Optimization: Experiments showed that using 9 blocks for local processing and a local detection coefficient of 1.2 provided a good balance between restoration effect and processing time.

Limitations and Future Work

  • Negative Impact in Refined Categories: While overall accuracy improved, the paper notes a slight increase in negative impact in the more refined four-category classification, particularly in the "Cancer" category. Physicians reported a tendency towards conservative diagnoses due to insufficient brightness after SR removal, suggesting a need for further refinement in brightness and contrast handling.
  • Real-time Application: The authors suggest that the method could be enhanced for real-time application during colposcopy, allowing physicians to toggle SR elimination and compare images dynamically.
  • Generalizability: While the method shows promise for other endoscopic images, further validation on diverse datasets would be beneficial.
  • Data Availability: The processed data required to reproduce the findings cannot be shared at this time as it is part of an ongoing study.

Summary updated

Abstract

This method can effectively eliminate the SR regions in the colposcopy image and present a satisfactory visual effect. <i>Significance</i>. As a preprocessing method for computer-aided diagnosis systems, it can also improve physicians' accuracy in clinical diagnosis.
BibTeX
@article{wang2021integration,
  author = {Xiaoxia Wang and Ping Li and Yuchun Lv and Huifeng Xue and Tianxiang Xu and Yongzhao Du and Peizhong Liu},
  title = {{Integration of Global and Local Features for Specular Reflection Inpainting in Colposcopic Images}},
  journal = {Journal of Healthcare Engineering},
  year = {2021},
  doi = {10.1155/2021/5401308},
}
Figure