TL;DR
This paper introduces a novel homography-based visual servoing system for robotic endoscopes that enables semi-autonomous manipulation without requiring explicit camera or tool positions, or depth information. By focusing on image-to-image transformations (homographies) and respecting a programmable Remote Center of Motion (RCM), the system allows surgeons to define and navigate between desired views, simplifying complex surgical tasks.
What Problem Does This Paper Address?
Minimally Invasive Surgery (MIS) relies heavily on endoscopic visualization, but manual control can be taxing for surgeons, leading to fatigue and potential errors. Existing robotic endoscope holders and visual servoing methods often depend on knowing precise tool and camera positions, or require depth information, which can be difficult to obtain reliably in dynamic surgical environments. These methods can also be sensitive to calibration errors and patient movement. The paper addresses the need for a more robust and intuitive visual servoing system that can automate camera control by focusing on desired visual scenes rather than precise geometric points, while also adhering to the crucial Remote Center of Motion (RCM) constraint essential for surgical manipulation.
Methods
The core innovation lies in a homography-based visual servoing task. Instead of tracking specific points, the system learns to map the current endoscopic view to a desired view using homographies. A homography is a transformation that relates corresponding points in two images of the same scene taken from different viewpoints. This allows the system to servo towards a desired image appearance rather than a specific geometric configuration.
This homography-based task is integrated with a programmable Remote Center of Motion (RCM). The RCM is a crucial constraint in MIS, ensuring that the instrument enters the body at a fixed point (the trocar) while allowing for manipulation within the body. The paper extends existing RCM control methods by incorporating the homography-based visual servoing objective into a composite Jacobian framework. This allows the robot to simultaneously satisfy the RCM constraint and achieve the desired visual target.
The system uses a graph-based representation of desired views. Surgeons can manually capture and store key views, building a graph where nodes are images and edges represent transitions between them. The system can then automatically plan and execute a path through this graph to reach a selected target view.
Datasets
The paper evaluates its method on an abdominal phantom using a Szabo Pelvic Trainer and a Kyoto Kagaku colon rectum tube to simulate a laparoscopic view. This setup serves as a custom benchmark for evaluating the system's performance in clinically relevant scenarios. No publicly available datasets were used for training or evaluation.
Key Insights and Ideas
- Scene-centric vs. Point-centric Servoing: The shift from tracking specific points to matching image appearances (via homographies) is a significant departure. This makes the system more robust to variations in lighting, minor occlusions, and the absence of easily identifiable features, as long as the overall scene appearance can be matched.
- Programmable RCM Flexibility: Using a programmable RCM, as opposed to a fixed mechanical one, allows for greater adaptability and can be integrated with other robotic functionalities.
- Graph-based View Navigation: Building a graph of desired views provides an intuitive way for surgeons to define targets and for the system to plan complex, multi-step movements. This simplifies the user interaction significantly.
- Independence from Depth and Explicit Positions: The ability to perform visual servoing without explicit camera or tool positions, or depth information, addresses a major limitation of previous methods and makes the system more practical for real-world surgical applications.
Key Results
The system demonstrated successful visual servoing in three scenarios on the abdominal phantom:
- Servoing between arbitrary views: The system achieved convergence with a final camera tip deviation of 0.4 mm and an RCM deviation from the trocar of 0.8±0.8 mm.
- Servoing under tool motion: Despite simulated tool movement, the system converged to desired views with pixel accuracy, showing a final camera position deviation of 1.4 mm.
- Servoing after phantom repositioning: The system successfully readjusted the view after phantom rotation and tilting, achieving submillimeter camera position convergence and minimizing visual error to pixel range.
The integral term in the controller was found to be crucial for removing steady-state errors in image alignment. The system operates at a frame rate of approximately 30 Hz for image processing and 100 Hz for RCM control.
Limitations and Future Work
- Initialization of RCM: The RCM still requires initialization, which is a standard procedure but a necessary step.
- Homography Estimation Noise: The homography estimation can be noisy due to feature sparseness, requiring filtering.
- Redundant DOF: The robot has one redundant DOF, which means it doesn't always return to its exact initial configuration. The authors suggest this could be addressed by incorporating joint state objectives into the Jacobian's nullspace.
- Future Work:
- Incorporating force-torque sensing to update the RCM.
- Developing more robust homography estimation methods, potentially using deep learning, that are invariant to object motion and feature sparseness.
- Investigating the use of joint state objectives to manage redundant DOFs.
Summary updated