TL;DR
This paper presents the first fully automated system for clearing blood from a surgical field using robotic suction, a critical step in hemostasis. It introduces a novel image-based blood flow detection and tracking algorithm that leverages temporal information and a trajectory generation technique to guide a suction tool, demonstrating effective blood removal in simulated and real-world scenarios.
What Problem Does This Paper Address?
Surgical automation has made strides in many areas, but a crucial and challenging task—hemostasis (controlling bleeding)—remains largely manual. Hemostasis involves managing bleeding from various sources, from small capillaries to larger vessels, with unpredictable flow rates and trajectories. When bleeding is severe, it can obscure the surgical field, making it difficult for surgeons to operate. This paper tackles the initial, yet vital, stage of hemostasis: clearing the surgical field of flowing blood to improve visibility and enable subsequent steps like vessel repair.
Methods
The core of the proposed solution lies in two main components:
- Probabilistic Blood Flow Detection and Tracking: Instead of relying solely on static image features, this method uses optical flow to detect motion. A pre-trained convolutional neural network (CNN) estimates optical flow, and its magnitude is used as an initial indicator of blood. To overcome the noise and transient nature of this detection, a Hidden Markov Model (HMM) is employed. This HMM fuses temporal information from consecutive frames, creating a probabilistic map of blood presence. This temporal filtering is key to distinguishing actual blood flow from other scene elements and handling the challenging lighting and reflective surfaces often found in surgical environments.
- Autonomous Suction Trajectory Generation: Once blood regions are identified, a trajectory generation algorithm guides a robotic suction tool. The algorithm determines start and end points based on the "age" of the detected blood (newer blood near the source, older blood downstream). To ensure efficient clearing, the trajectory is planned to move upstream towards the bleeding source while staying centered within the blood flow. This is achieved by incorporating a "clearance reward" into a Dijkstra's algorithm pathfinding, which penalizes paths that deviate from the center of the blood stream.
Datasets
- Simulated Scenes: Six simulated scenes of flowing blood were generated using Unity3D's particle-based fluid dynamics. 61 image frames were extracted per scene, with ground-truth masks of the blood region manually annotated.
- In-Vivo Surgical Scene: An 8-second video clip from a real thyroidectomy where a carotid artery rupture occurred was used. 10 evenly distributed frames were manually annotated for ground-truth blood region masks.
- Tissue Phantom: A silicone phantom with a cavity was constructed and filled with red-colored water to simulate blood. This was used for evaluating the complete automated suctioning solution.
Key Insights and Ideas
- Temporal Information is Crucial for Surgical Blood Detection: Relying solely on static color or optical flow magnitude is insufficient. The HMM-based temporal filtering significantly improves blood detection accuracy, especially in challenging surgical environments with stains and variable lighting.
- Optical Flow Magnitude, Not Orientation, is Key for Fluid Detection: The paper confirms that the magnitude of optical flow is a strong indicator of fluid motion, while its orientation provides less reliable information for this task.
- Upstream Suction with Centered Trajectories is Optimal: Simply moving towards the bleeding source isn't enough. The developed trajectory generation method, which prioritizes moving upstream and staying centered within the blood flow, is crucial for effectively clearing the surgical field.
- Robustness to Visual Clutter: The system demonstrated effectiveness even with red stains from previous ruptures, highlighting the power of the temporal tracking approach.
Key Results
- The complete pipeline (blood detection, tracking, and trajectory generation) achieved better performance than using only optical flow thresholding, as indicated by Intersection over Union (IoU) metrics.
- In simulated scenes, the system achieved an IoU of approximately 0.8-0.9, while in the in-vivo scene, it achieved an IoU of around 0.5-0.7. While lower than simulated, this is still considered a significant achievement given the challenging real-world conditions.
- The automated suction system successfully removed over 93% of injected liquid in a tissue phantom experiment across various injection points.
- The system demonstrated a fast reaction time, with detection and trajectory generation completing within 2.6 to 6.4 seconds, and the suction trajectory execution taking between 23.9 to 47.4 seconds.
Limitations and Future Work
- Limitations: The paper acknowledges that the in-vivo performance was lower than simulated, likely due to poorer lighting conditions and reflective surgical clamps. The current system focuses on clearing blood, not directly locating the bleeding source itself.
- Future Work: The authors plan to develop a more precise method for locating the bleeding source using particle-based motion models. They also intend to integrate surgical tool masking techniques to further enhance the robustness of the blood tracking framework, as hemostasis automation will likely involve multiple surgical tools.
Summary updated