Capsule Vision 2024 Challenge: Multi-Class Abnormality Classification for Video Capsule Endoscopy

Palak Handa, Amirreza Mahbod, Florian Schwarzhans, Ramona Woitek, Nidhi Goel, Deepti Chhabra, Shreshtha Jha, Manas Dhir, Deepak Gunjan, Jagadeesh Kakarla, Balasubramanian Raman

TL;DR

This paper introduces the Capsule Vision 2024 Challenge, a competition focused on multi-class abnormality classification in Video Capsule Endoscopy (VCE) images. The challenge aims to drive the development of advanced AI models to assist gastroenterologists in analyzing VCE videos more efficiently and accurately, addressing the growing burden of gastrointestinal diseases.

What Problem Does This Paper Address?

Gastrointestinal (GI) and liver diseases are a significant global health concern. Video Capsule Endoscopy (VCE) offers a non-invasive way to visualize the GI tract, but analyzing the vast number of frames (up to 1 million per procedure) is time-consuming and prone to human error. This manual review process can take hours per patient, leading to delays and potential misdiagnoses. The challenge aims to leverage Artificial Intelligence (AI) to automate the classification of various abnormalities within VCE images, thereby reducing the workload on clinicians and improving diagnostic precision.

Methods

The core of this paper is the description of the Capsule Vision 2024 Challenge itself. The challenge involved developing AI models to classify 10 distinct classes of abnormalities (angioectasia, bleeding, erosion, erythema, foreign body, lymphangiectasia, polyp, ulcer, worms, and normal) from VCE video frames. Participants were provided with training, validation, and testing datasets. The challenge rules emphasized using only the provided datasets for training and validation, with submissions evaluated on their performance on the unseen test set. The evaluation metric was a combined score of Mean AUC and Balanced Accuracy. The paper also details a pipeline for dataset development, involving data collection from multiple public and private sources, frame extraction, cropping, resizing, and splitting.

Datasets

The challenge utilized a combination of publicly available and private datasets for training, validation, and testing. The training and validation datasets were derived from sources like kvasir-capsule, SEE-AI project, and KID, along with a private AIIMS dataset. These datasets were processed to create a large dataset with 37,607 training frames and 16,132 validation frames, covering the 10 specified classes. The testing dataset was a high-quality, medically annotated private dataset from the Department of Gastroenterology and HNU, All India Institute of Medical Sciences, Delhi, comprising 4,385 VCE frames.

Key Insights and Ideas

  • The critical need for AI in VCE: The paper strongly highlights the current limitations of manual VCE analysis, emphasizing the time burden, human bias, and potential for missed abnormalities. This underscores the urgent need for AI-driven solutions.
  • Standardized benchmark for VCE abnormality classification: The challenge provides a much-needed standardized platform for researchers to develop and compare AI models for this specific task, fostering innovation and collaboration.
  • Multi-class classification complexity: The inclusion of 10 distinct classes, including subtle abnormalities and normal frames, presents a significant challenge, requiring models that can differentiate between a wide range of visual patterns.
  • Importance of robust dataset curation: The detailed description of the dataset development pipeline showcases the effort involved in creating a comprehensive and well-annotated dataset, crucial for training effective AI models.

Key Results

The challenge attracted approximately 150 teams, with 35 ultimately submitting results. After evaluation, 27 teams were qualified. The top-ranked team, "PuppyOps," achieved a combined metric of 0.6071809. Table II in the paper lists the rankings of the top 27 teams based on their performance on the test dataset, with "MedInfoLab IIT Hyderabad" and "WueVision" securing second and third place, respectively. Table III presents benchmarking results for different model architectures (VGG19, Xception, ResNet50V2, etc.) evaluated by the organizing team, with ResNet50V2 achieving the highest combined metric of 0.3597053 among these baseline models.

Limitations and Future Work

  • Limitations: The paper notes that for the benchmarking experiments (Table III), no data augmentation or fine-tuning of the AI models was performed. Some teams' submissions were disqualified due to incorrect formats or failure to meet submission requirements. The challenge itself is limited by the scope of the provided datasets and the specific classes defined.
  • Future Work: The challenge aims to promote the development of efficient AI models that can reduce analysis time while maintaining high diagnostic accuracy. The organizers encourage participants to use the provided resources to contribute to enhancing VCE abnormality classification. The paper implicitly suggests that future work could involve exploring more advanced AI architectures, incorporating temporal information from video sequences, and expanding the range of detectable abnormalities.

Summary updated

Abstract

We present the Capsule Vision 2024 Challenge: Multi-Class Abnormality Classification for Video Capsule Endoscopy. It is being virtually organized by the Research Center for Medical Image Analysis and Artificial Intelligence (MIAAI), Department of Medicine, Danube Private University, Krems, Austria and Medical Imaging and Signal Analysis Hub (MISAHUB) in collaboration with the 9th International Conference on Computer Vision & Image Processing (CVIP 2024) being organized by the Indian Institute of Information Technology, Design and Manufacturing (IIITDM) Kancheepuram, Chennai, India. This document describes the overview of the challenge, its registration and rules, submission format, and the description of the utilized datasets.
BibTeX
@misc{handa2024capsule,
  author = {Palak Handa and Amirreza Mahbod and Florian Schwarzhans and Ramona Woitek and Nidhi Goel and Deepti Chhabra and Shreshtha Jha and Manas Dhir and Deepak Gunjan and Jagadeesh Kakarla and Balasubramanian Raman},
  title = {{Capsule Vision 2024 Challenge: Multi-Class Abnormality Classification for Video Capsule Endoscopy}},
  year = {2024},
  eprint = {2408.04940},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.04940},
}

Loading related content...

Figure