Introduction
In laparoscopic cholecystectomy, the surgical removal of the gallbladder, preventing bile duct injuries is a paramount safety concern. Surgeons rely on the Critical View of Safety (CVS)—a specific anatomical visualization involving the identification of the cystic duct and artery—to ensure safe dissection. Automating the recognition of CVS through artificial intelligence represents a "holy grail" for surgical assistance systems. However, developing such systems is hindered by two primary challenges: the exorbitant cost of pixel-wise spatial annotations and the inherent subjectivity in expert evaluations, which complicates binary classification.
This paper presents CVS-AdaptNet, a novel multi-modal framework designed to overcome these barriers. By leveraging the textual knowledge embedded in surgical foundation models and Large Language Models (LLMs), CVS-AdaptNet achieves high-accuracy CVS recognition using only image-level labels, bypassing the need for labor-intensive manual segmentation.
Methodology: CVS-AdaptNet
The core of the proposed approach is the adaptation of multi-modal surgical models to the fine-grained task of CVS assessment. Unlike general-purpose models, CVS-AdaptNet is tailored to handle the nuances of surgical anatomy through several key strategies:
LLM-Augmented Prompts
Standard classification models often rely on simple class labels. In contrast, CVS-AdaptNet utilizes GPT-4 to generate diverse "positive" and "negative" textual descriptions for each of the three CVS criteria. For instance, instead of a simple "Criterion 1" tag, the model is trained with detailed descriptions of what seeing the cystic duct and artery looks like, as well as descriptions of scenes where these structures are obscured. This approach provides the model with a richer semantic context, helping it learn anatomical relationships more effectively.
KL-Divergence Loss and Multi-Label Reformulation
Traditional contrastive learning often forces a hard match between images and text. However, surgical scenes are often ambiguous. CVS-AdaptNet employs Kullback-Leibler (KL) divergence loss, which allows the model to represent the "softness" or uncertainty of surgical labels, better reflecting the reality of expert disagreement. Furthermore, the framework reformulates the task into a multi-label setting, acknowledging that multiple CVS criteria can be satisfied simultaneously within a single frame.
Experimental Setup
The researchers validated their framework using the Endoscapes-CVS201 dataset, which contains 11,090 frames annotated for the three Strasberg CVS criteria. The model's backbones—PeskaVLP, SurgVLP, and HecVL—were pre-trained on the SVL dataset, a large-scale collection of surgical lecture videos. This pre-training ensures that the encoders already possess a foundational understanding of surgical environments before being fine-tuned for the CVS task.
Results and Analysis
The experimental results demonstrate that CVS-AdaptNet significantly outperforms standard image-only baselines. The framework achieved a mean Average Precision (mAP) of 57.6, a 6-point improvement over the ResNet50 baseline (51.5 mAP).
Key Insights
- Text as a Proxy for Spatial Data: The use of detailed clinical descriptions allows the model to learn spatial relationships implicitly. By describing the anatomy, the text guides the image encoder to focus on relevant regions without requiring explicit bounding boxes or masks.
- The Power of Negation: The inclusion of negative prompts—descriptions of what is not present—was crucial. It significantly improved the model's ability to distinguish between nearly-complete and fully-achieved CVS criteria.
- Foundation Model Alignment: Among the tested backbones, PeskaVLP exhibited the strongest alignment between image and text features, suggesting it is particularly well-suited for high-stakes clinical adaptation.
Conclusion and Future Work
CVS-AdaptNet represents a significant step toward scalable surgical AI by reducing the reliance on expensive annotations. While it narrows the performance gap, a "segmentation gap" still exists; models that utilize explicit segmentation masks currently achieve higher performance (~67.3 mAP). This suggests that while text is a powerful supervisor, knowing the exact spatial location of structures remains a distinct advantage.
Future research will focus on "generalized decoding," aiming to combine the flexibility of multi-modal text prompts with the precision of automated segmentation techniques. This hybrid approach could potentially offer the best of both worlds: the ease of image-level training with the accuracy of spatial awareness.
Code available at: https://github.com/CAMMA-public/CVS-AdaptNet
Summary updated