Abstract
Video capsule endoscopy (VCE) is an important technology with many advantages (non-invasive, representation of small bowel), but faces many limitations as well (time-consuming analysis, short battery lifetime, and poor image quality). Artificial intelligence (AI) holds potential to address every one of these challenges, however the progression of machine learning methods is limited by the availability of extensive data. We propose Galar, the most comprehensive dataset of VCE to date. Galar consists of 80 videos, culminating in 3,513,539 annotated frames covering functional, anatomical, and pathological aspects and introducing a selection of 29 distinct labels. The multisystem and multicenter VCE data from two centers in Saxony (Germany), was annotated framewise and cross-validated by five annotators. The vast scope of annotation and size of Galar make the dataset a valuable resource for the use of AI models in VCE, thereby facilitating research in diagnostic methods, patient care workflow, and the development of predictive analytics in the field.
Introduction
Video Capsule Endoscopy (VCE) is a marvel of non-invasive diagnostics: a patient swallows a pill-sized camera that records their entire digestive tract. However, the procedure generates a "needle in a haystack" problem. Medical staff must often watch hours of footage to find a few seconds of relevant pathology. While Artificial Intelligence (AI) is the obvious solution for automating this review, the field has been hamstrung by a lack of diverse, high-quality data. Existing VCE datasets are typically small, limited to specific pathologies, or sourced from a single hardware system.
To bridge this gap, we introduce Galar, a massive multi-center benchmark designed to provide a robust foundation for training AI to automate the grueling process of manual VCE review.
Dataset Composition and Comparison
The primary contribution of this work is the Galar dataset itself. It comprises over 3.5 million frames sourced from 80 videos across two different medical centers in Germany. To ensure hardware agnosticism, the data includes footage from both Olympus™ and various PillCam™ models.
As shown in the comparison above, while some datasets like Rhode Island contain more raw images, Galar provides a significantly higher density of labels (29 distinct classes). These labels are categorized into technical quality (e.g., bubbles, dirt), anatomical landmarks, and 19 distinct pathological findings, allowing models to understand medical context rather than just identifying disease.
Methodology
The authors developed Galar using a rigorous, multi-step pipeline:
- Data Sourcing: Collection of 80 videos from two centers using distinct hardware systems to address the "domain shift" problem.
- Annotation: Using the open-source CVAT tool, a team of five annotators labeled every single frame. Frames marked as "unrecognizable" were reviewed by a senior expert with 10 years of experience.
- Multi-Label Schema: Implementation of a system covering technical, anatomical, and pathological aspects.
- Technical Validation: Training baseline ResNet-50 models to perform multi-label and multi-class classification tasks to prove the dataset's utility.
Key Results
The baseline ResNet-50 models demonstrated that Galar is a high-quality training resource, particularly for anatomical and technical classification.
As detailed in the results, the models performed exceptionally well on large anatomical structures, achieving 93% accuracy for the stomach and 92% for the small intestine. Technical quality classification for "dirt" and "bubbles" reached accuracies of 88% and 82%, respectively. However, detecting rare pathologies remains a challenge; while the AUROC for "blood" was a respectable 0.87, the F1-score was only 0.14, reflecting the massive class imbalance inherent in clinical VCE data.
Limitations and Future Work
While Galar is the most comprehensive dataset to date, it still faces the "long tail" problem common in medical imaging. Certain rare pathologies did not occur in this specific set of 80 videos. Future work will focus on imbalanced learning and more complex loss functions to help AI catch these critical but rare findings. Released under a CC BY 4.0 license, Galar invites the global community to apply advanced architectures like Transformers to improve upon these baseline results.
Summary updated