Cross-Dataset Adaptation for Instrument Classification in Cataract Surgery Videos

Jay N. Paranjape, Shameema Sikder, Vishal M. Patel, S. Swaroop Vedula

Abstract

Surgical tool presence detection is a cornerstone of intra-operative assistance and post-operative analysis in Surgical Data Science (SDS). However, state-of-the-art models often suffer from significant performance degradation when deployed across different clinical environments—a phenomenon known as domain shift. This paper addresses the challenge of domain shift in cataract surgery by introducing the Barlow Adaptor, an end-to-end Unsupervised Domain Adaptation (UDA) framework. Central to this method is the Barlow Feature Alignment Loss (BFAL), which aligns feature distributions across domains while minimizing redundancy. Experimental results on the CATARACTS and D99 datasets demonstrate that the Barlow Adaptor outperforms existing UDA methods by approximately 6%, providing a robust solution for cross-dataset instrument classification without requiring target-domain labels.

Introduction

As deep learning models move from research benchmarks to real-world operating rooms, the "brittleness" of these systems becomes a primary concern. In cataract surgery, variations in camera hardware, lighting conditions, and surgical protocols create a distinct "domain shift" between datasets. A model trained on one dataset (the source) often fails to generalize to another (the target), necessitating expensive and time-consuming manual labeling for every new surgical site.

To mitigate this, Unsupervised Domain Adaptation (UDA) seeks to leverage labeled source data and unlabeled target data to build generalizable models. This paper proposes the Barlow Adaptor, which utilizes principles from self-supervised learning to create domain-agnostic feature representations.

The Barlow Adaptor Framework

Barlow Feature Alignment Loss (BFAL)

Inspired by the "Barlow Twins" self-supervised learning paradigm, the authors propose BFAL to address the limitations of traditional distribution matching. BFAL projects features from both the source and target domains into a high-dimensional space and computes a cross-correlation matrix. The loss function pursues two primary objectives:

  1. Alignment: It pushes the diagonal elements of the cross-correlation matrix toward 1, ensuring that features from both domains are closely aligned.
  2. Redundancy Reduction: It pushes the off-diagonal elements toward 0. This forces the model to learn independent, non-redundant features, preventing it from latching onto domain-specific noise or artifacts that do not generalize.

Hybrid Loss Strategy

The Barlow Adaptor does not rely on BFAL in isolation. It employs a multi-faceted loss strategy to ensure both discriminative power and domain invariance:

  • Cross-Entropy Loss: Applied to the labeled source data to maintain high classification accuracy.
  • Deep CORAL: Used to align the second-order statistics (covariances) of the source and target distributions.
  • BFAL: Used to refine the feature alignment and reduce redundancy.

This architecture is backbone-agnostic, demonstrating effectiveness across both traditional Convolutional Neural Networks (ResNet50) and modern Vision Transformers (ViT).

Experimental Setup

Datasets and Mapping

The study utilizes two major cataract surgery benchmarks:

  • CATARACTS: 50 videos with framewise annotations for 21 instruments.
  • D99: 105 videos with annotations for 25 instruments.

Because the datasets use different labeling conventions, the authors identified 14 common instrument classes (e.g., Phacoemulsifier Handpiece, Bonn Forceps, Irrigation) to facilitate cross-dataset evaluation.

Implementation Details

One significant advantage of the Barlow Adaptor is its computational efficiency. While many contrastive learning methods require massive batch sizes to be effective, this framework achieved superior results with a batch size of only 16, making it accessible for standard research hardware.

Results and Discussion

Performance Gains

In the D99 → CATARACTS transfer task, the Barlow Adaptor achieved a 35.1% macro accuracy using a ResNet50 backbone. This represents a substantial improvement over the baseline "Source Only" model (27.9%), which lacked any adaptation. Furthermore, the method outperformed established SOTA UDA techniques, such as Domain-Adversarial Neural Networks (DANN) and Cross Domain Transformers (CDTrans), by a margin of roughly 6%.

Ablation Insights

Ablation studies confirmed that the combination of BFAL and CORAL is essential. Aligning both the correlations and covariances of the feature sets provides a more comprehensive bridge between domains than using either method alone. The "tug-of-war" between classification loss (which seeks distinct features) and alignment loss (which seeks similar features) results in a feature set that captures the essential characteristics of surgical tools regardless of the imaging environment.

Conclusion and Future Work

The Barlow Adaptor represents a significant step toward deploying robust AI in the surgical suite. By focusing on redundancy reduction and feature alignment, the method overcomes the common pitfalls of domain shift in cataract surgery.

Despite these gains, a performance gap remains between UDA models (~35%) and models trained directly on target data (~55%). Future research will focus on addressing multi-label classification—where multiple tools are used simultaneously—and incorporating temporal dependencies in surgical video to further improve classification accuracy in complex operative scenarios.

Summary updated

Abstract

Surgical tool presence detection is an important part of the intra-operative and post-operative analysis of a surgery. State-of-the-art models, which perform this task well on a particular dataset, however, perform poorly when tested on another dataset. This occurs due to a significant domain shift between the datasets resulting from the use of different tools, sensors, data resolution etc. In this paper, we highlight this domain shift in the commonly performed cataract surgery and propose a novel end-to-end Unsupervised Domain Adaptation (UDA) method called the Barlow Adaptor that addresses the problem of distribution shift without requiring any labels from another domain. In addition, we introduce a novel loss called the Barlow Feature Alignment Loss (BFAL) which aligns features across different domains while reducing redundancy and the need for higher batch sizes, thus improving cross-dataset performance. The use of BFAL is a novel approach to address the challenge of domain shift in cataract surgery data. Extensive experiments are conducted on two cataract surgery datasets and it is shown that the proposed method outperforms the state-of-the-art UDA methods by 6%. The code can be found at https://github.com/JayParanjape/Barlow-Adaptor
BibTeX
@inproceedings{paranjape2023cross,
  author = {Jay N. Paranjape and Shameema Sikder and Vishal M. Patel and S. Swaroop Vedula},
  title = {{Cross-Dataset Adaptation for Instrument Classification in Cataract Surgery Videos}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  url = {https://openreview.net/forum?id=JGqZRvxWnS9},
}
Figure