IDDF2021-ABS-0206 A new computer-assisted diagnosis scoring system based on deep learning for predicting inflammatory activity from patients with ulcerative colitis
Hongzhi Xu, Yanyun Fan, Ruochen Mu, Yinghao Zhang, Chenxi Xie, Lupeng Liu, Lin Wang, Jianlin Ren, Liansheng WangClinical2021 · DOI
Endoscopic evaluation is critical for managing Ulcerative Colitis (UC), yet traditional assessment methods suffer from significant inter-observer variability. Clinicians' training and experience levels often lead to inconsistent scoring. Furthermore, standard metrics like the Mayo score and the Ulcerative Colitis Endoscopic Index of Severity (UCEIS) typically focus on the most severe lesions, failing to capture the full spatial distribution of inflammation along the colon. This study introduces an automated computer-assisted diagnosis (CAD) system designed to provide objective, continuous scoring from both endoscopic images and full-length videos.
Methodology
The researchers developed a deep learning framework utilizing a ResNet50 convolutional neural network (CNN) backbone, pre-trained on ImageNet. The system architecture integrates three specialized sub-networks:
Clarity Cleaning: Automatically filters out poor-quality or obscured frames from video feeds.
Similarity Judgment: Ensures consistent feature extraction across different colonic segments.
Score Prediction: Calculates Mayo and UCEIS scores based on learned features.
A novel feature of this CAD system is the generation of a "ribbon" visualization. This graphical output maps the colon's shape and uses color-coding to represent the location and severity of inflammatory activity, offering a comprehensive spatial overview of the disease state. The model was trained on a dataset of 5,875 endoscopic images and validated using 16 full-length videos (comprising 1,360 local data points across 100 intestinal segments) collected between 2017 and 2021.
Key Results
The CAD system demonstrated high diagnostic accuracy and strong agreement with expert clinical assessments, as measured by the Quadratic Weighted Kappa (QWK) coefficient.
Mayo Score Prediction: The system achieved 90.2% accuracy with a QWK of 0.906 (95% CI, 0.880–0.933).
UCEIS Sub-score Prediction:
Vascular Pattern: 90.7% accuracy (QWK 0.887).
Erosions and Ulcers: 84.6% accuracy (QWK 0.876).
Bleeding: 77.7% accuracy (QWK 0.816).
In video analysis, the system successfully categorized localized intestinal tissue into four severity ranges, showing a distribution that closely aligned with clinical diagnoses.
Discussion and Conclusion
This deep learning-based scoring system represents a significant advancement over traditional "peak severity" snapshots. By analyzing full-length endoscopic videos and providing a weighted calculation of inflammation, the system offers a more holistic view of UC activity.
While the results are promising, future work should focus on expanding the video validation set beyond the initial 16 cases and testing the model across multi-center data to ensure generalizability across different endoscopic hardware. Ultimately, this tool has the potential to standardize UC assessment, reduce observer bias, and assist clinicians in more precise disease monitoring and treatment planning.
Summary updated
Abstract
<h3>Background</h3> Endoscopy is increasingly important for the evaluation of patients with ulcerative colitis (UC). However, there were considerable differences in endoscopic assessment because of the endoscopists’ training experience, not only that, existing endoscopy scoring methods cannot reflect the inflammation’s details of the full-length endoscopic video; therefore, we sought to develop an automatic scoring system using deep learning technology for consistent and objective of endoscopic images and videos from patients with UC. <h3>Methods</h3> ResNet50 pre-trained on ImageNet was selected as our CNN main framework. We collected 5,875 endoscopic images and 16 full-length videos from patients with UC who underwent colonoscopy from January 2017 through March 2021as the dataset. The scoring system was trained by these images and could predict the Mayo score and UCEIS score by the feature it has learned on the training set. In addition, by using the predicted outcome, the CAD could show a ribbon in the shape of the colon which can reflect the conditions of the inflammation (location, severity). The scoring system is composed of clarity cleaning, similarity judgement and score prediction networks. This corresponding relation which combines the exiting endoscopy scores prediction, helps the system to output a new score with weighted calculation, as well as the lesions’ location and severity ratio. <h3>Results</h3> Two clinical reviewed our consistency statistics. The primary evaluation metric we used is quadratic weighted kappa (QWK). The trained CNN got high levels with 90.2% accuracy in the Mayo-scored task, while the QWK coefficient was 0.906 (95%CI, 0.880-0.933). The metrics of UCEIS-scored task were inspiring of which accuracy were 90.7%, 84.6%, 77.7%, QWK coefficient were 0.887 (95%CI, 0.860-0.915), 0.876 (95%CI, 0.844-0.907), 0.816 (95%CI, 0.739-0.892) for vascular pattern, erosions-and- ulcers and bleeding. Meanwhile, the CNN identified 16 full-length endoscopy videos and evaluated 1,360 local information from 100 intestine segments. The results of these localized intestinal tissue marked by the scored system were distributed in four ranges, and the quantities were 772(0.0-0.4), 124(0.72-2.35), 206(3.2-4.0), 258(5.2-8.4). It performed desirable compared with the clinical diagnosis. <h3>Conclusions</h3> The results of objective evaluations of UC based on this scoring system appeared acceptable. Compared with the previous scored methods (only refer to the most severe lesion and ignored other mild parts), this work is of clinical significance and will help clinicians by supplying a good capacity in identifying endoscopy data and stimulating the colon’s inflammatory environment.
BibTeX
@inproceedings{xu2021iddf,
author = {Hongzhi Xu and Yanyun Fan and Ruochen Mu and Yinghao Zhang and Chenxi Xie and Lupeng Liu and Lin Wang and Jianlin Ren and Liansheng Wang},
title = {{IDDF2021-ABS-0206 A new computer-assisted diagnosis scoring system based on deep learning for predicting inflammatory activity from patients with ulcerative colitis}},
booktitle = {Clinical},
year = {2021},
doi = {10.1136/gutjnl-2021-iddf.174},
}