Abstract
We introduce a novel 3 degrees-of-freedom based robotic colonoscopy system that performs the necessary movements for colonoscopy while working within the movement range of a flexible colonoscope (FC). In addition, we have developed deep learning models to generate motor control signals directly from input images without the need for motor control signal labels. The first presented model comprises a deep learning algorithm for predicting steering points and an image-based visual servo control (IBVS) algorithm for generating the motor control signal. The experiments showed that the proposed model’s cecal intubation time (CIT) and rate (CIR) are comparable to those of human operators, despite requiring a shorter training time. Furthermore, we propose a model that replaces the IBVS algorithm with a deep learning algorithm that does not rely on rotation angles. The second model showed similar CIT (165s) and CIR (92%) compared to the first model. Finally, the last model, which solely comprises a single deep learning algorithm, demonstrates a reduction in CIT (127s) and an increase in CIR (96%), resulting in reduced physical demand for operators, improved safety, and shorter patient recovery time.
Introduction
Colorectal cancer remains a leading cause of death globally, yet colonoscopies—the gold standard for early detection—are notoriously difficult to perform. The procedure requires a high level of manual dexterity to manage simultaneous pushing, pulling, rotating, and steering of a flexible tube through a complex, winding environment. This complexity leads to significant physical strain and "doctor fatigue" for endoscopists. While robotic and magnetic-assisted systems have been proposed to help, they often suffer from slow intubation times, low success rates, and unreliable lumen detection. This paper addresses these challenges by introducing a 3 degrees-of-freedom (DoF) robotic system and an end-to-end deep learning model that can autonomously navigate a colonoscope.
System Architecture and Methods
The researchers developed a hardware-software duo: the 3-RCS (3 Degrees-of-Freedom Robotic Colonoscopy System) and a series of evolving deep learning control strategies.
Hardware (3-RCS)
The custom rig features two primary mechanisms: a Feeding Mechanism, which uses four rollers to move the scope forward and backward, and a Steering Mechanism, which utilizes two motors to pull internal wires for two-directional bending.
Control Evolution
Three distinct methods were evaluated for autonomous navigation:
- Method A: A CNN identifies the "steering point" in the image and calculates collision probability. It then uses a traditional Image-Based Visual Servo (IBVS) algorithm to generate motor commands based on real-time pulley angle feedback.
- Method B: This method replaces the IBVS with a 1DCNN-LSTM model that predicts steering signals directly from the steering points, eliminating the need for noisy real-time pulley angle feedback caused by cable slip and friction.
- Method C (End-to-End): The final model uses a MobileNetV2 backbone to process raw images and directly output all necessary control signals: steering velocities, feeding speed, and collision probability.
Experimental Setup and Data Collection
The system was evaluated using a phantom colon model within a ROS-based environment. Training data was generated through an iterative process involving manual trials and autonomous runs.
- Dataset 1: 52,771 images from 27 manual trials, hand-labeled with steering points and classified as "wall" or "non-wall."
- Dataset 2: Inputs and outputs recorded from Method A during 12 autonomous runs to train the 1DCNN-LSTM.
- Dataset 3: Images and control signals from 30 successful autonomous runs of Method A, used to train the final End-to-End (E2E) model.
Key Results
The End-to-End model (Method C) was the clear winner across 50 experimental trials:
- Efficiency: It achieved a mean Cecal Intubation Time (CIT) of 127 seconds, which is significantly faster than novice human operators (166 seconds) and Method B (165 seconds).
- Success Rate: It reached the cecum in 96% of trials, demonstrating high reliability.
- Stability: Method C showed the lowest standard deviation in time, meaning its performance was more consistent and predictable than the other methods.
- Sectional Performance: While all methods struggled most with the rectosigmoid (the sharpest turn in the colon), Method C navigated it in 50 seconds, compared to 83 seconds for human novices.
Conclusion and Future Work
By moving beyond traditional "darkest point" steering and removing reliance on noisy physical feedback, the proposed end-to-end model provides a robust solution for autonomous colonoscopy. However, a primary failure mode remains "elongation," where the colon stretches as the scope head becomes stuck. Future research will focus on integrating hardware sensors, such as accelerometers and gyroscopes, to detect these states and exploring Vision Transformers (ViTs) to further enhance navigation accuracy.
Summary updated