Abstract
Purpose: Advances in deep learning have resulted in effective models for surgical video analysis; however, these models often fail to generalize across medical centers due to domain shift caused by variations in surgical workflow, camera setups, and patient demographics. Recently, object-centric learning has emerged as a promising approach for improved surgical scene understanding, capturing and disentangling visual and semantic properties of surgical tools and anatomy to improve downstream task performance. In this work, we conduct a multi-centric performance benchmark of object-centric approaches, focusing on Critical View of Safety assessment in laparoscopic cholecystectomy, then propose an improved approach for unseen domain generalization. Methods: We evaluate four object-centric approaches for domain generalization, establishing baseline performance. Next, leveraging the disentangled nature of object-centric representations, we dissect one of these methods through a series of ablations (e.g. ignoring either visual or semantic features for downstream classification). Finally, based on the results of these ablations, we develop an optimized method specifically tailored for domain generalization, LG-DG, that includes a novel disentanglement loss function. Results: Our optimized approach, LG-DG, achieves an improvement of 9.28% over the best baseline approach. More broadly, we show that object-centric approaches are highly effective for domain generalization thanks to their modular approach to representation learning. Conclusion: We investigate the use of object-centric methods for unseen domain generalization, identify method-agnostic factors critical for performance, and present an optimized approach that substantially outperforms existing methods.
Introduction
A significant bottleneck in the deployment of surgical AI is the "domain shift" problem. Deep learning models trained at one clinical site often experience a sharp decline in performance when deployed at another due to variations in camera equipment, lighting conditions, and surgical techniques. This lack of generalization is particularly problematic for critical tasks such as the automated assessment of the Critical View of Safety (CVS) during laparoscopic cholecystectomy. While CVS assessment is essential for preventing bile duct injuries, current models struggle to generalize to new medical centers without costly and labor-intensive manual re-annotation of data. This paper addresses Domain Generalization (DG), aiming to develop models that can be trained on a source center and perform effectively on an unseen target center.
Methodology: The LG-DG Approach
The proposed solution, LG-DG (Latent Graph - Domain Generalization), leverages Object-Centric Learning. Rather than processing images as flat grids of pixels, the scene is represented as a Latent Graph where nodes correspond to specific objects (e.g., cystic duct, liver, surgical tools) and edges represent their spatial and geometric relationships.
Through extensive ablation studies, the authors identified that semantic features—the identity and location of objects—are significantly more robust across different hospitals than visual features, such as texture and color, which are highly domain-dependent. To exploit this, LG-DG incorporates a novel disentanglement loss function. This function encourages the model to learn representations that remain effective for classification even when visual textures are masked or noisy, forcing the model to rely on the most stable, domain-invariant information in the scene.
Datasets and Domain Shift
The study evaluates cross-continental generalization using two distinct datasets:
- Endoscapes2023: Comprising 201 laparoscopic cholecystectomy videos from Strasbourg, France.
- Endoscapes-WC70: A new dataset consisting of 70 videos from West China Hospital in Sichuan, China, featuring annotations for CVS criteria and segmentation masks.
The visual gap between these datasets is substantial, involving different aspect ratios, color distributions, and fields of view, providing a rigorous test for zero-shot domain transfer.
Results and Key Insights
The experimental results demonstrate the superiority of object-centric graph representations for domain generalization:
- Performance Gains: LG-DG achieved a 33.34 mAP (mean Average Precision) in the zero-shot setting, representing a 9.28% improvement over the previous state-of-the-art baseline.
- Robustness of Graph Representations: Standard "black-box" CNNs like ResNet50 performed poorly (12.77 mAP), while all object-centric methods maintained performance above 23 mAP, suggesting that modular scene representations are inherently more robust to domain shift.
- The Reconstruction Trap: Interestingly, the authors found that auxiliary reconstruction tasks—often used to aid learning—can be detrimental to domain generalization as they force the model to encode domain-specific textures rather than invariant semantic relationships.
Conclusion
This work demonstrates that object-centric methods are highly effective for unseen domain generalization in surgical video analysis. By disentangling semantic relationships from domain-specific visual noise, the LG-DG approach substantially outperforms existing methods. Future work will explore Unsupervised Domain Adaptation (UDA) to further bridge the generalization gap by leveraging unlabeled data from target domains.
Summary updated