Introduction
Computer vision applications in the operating room (OR) offer significant benefits, such as monitoring radiation exposure and analyzing surgical workflows. However, deploying these systems is hindered by two primary challenges: privacy concerns and the high cost of data annotation. High-resolution video recording is often restricted due to privacy regulations for patients and staff. While low-resolution recording is a viable alternative, standard models trained on general datasets (e.g., COCO) often fail in the specialized, blurry environment of an OR. Furthermore, annotating OR data is labor-intensive, often requiring up to 90 minutes per image.
The AdaptOR Framework
To address these challenges, we present AdaptOR, an unsupervised domain adaptation (UDA) framework designed to transfer knowledge from labeled "in-the-wild" datasets to unlabeled OR environments. Built upon a Mask R-CNN backbone, AdaptOR introduces several key innovations:
Disentangled Feature Normalization (DFN)
Standard models typically use uniform normalization parameters. AdaptOR employs separate Group Normalization layers for the source and target domains. This "disentanglement" allows the model to learn domain-specific features independently, preventing statistical interference between disparate environments like a public park and a sterile OR.
Transformation Equivariant Constraints
The framework utilizes a Mean-Teacher architecture. A "Teacher" model processes weakly augmented OR images to generate pseudo-labels, which a "Student" model then attempts to predict on strongly augmented versions of the same images. To ensure pseudo-label reliability, geometric constraints (such as chirality) are applied: if an image is flipped, the corresponding keypoints must also flip consistently.
Privacy-Preserving Training
AdaptOR is specifically designed for robustness against heavy downsampling. By incorporating "strong-resize" augmentations that downsample images by up to 12x during training, the model learns to extract meaningful features from blurred inputs that effectively anonymize individuals.
Experimental Setup
The framework was evaluated using three datasets:
- COCO: The source domain, providing 57k labeled images of people in everyday settings.
- MVOR+: A target domain dataset featuring real surgical interventions.
- TUM-OR-test: Simulated clinical activities re-annotated with 2D keypoints for evaluation.
Results and Discussion
AdaptOR demonstrates significant improvements over source-only models and existing UDA baselines.
Robustness to Low Resolution
On 12x downsampled images—where human identification becomes difficult—AdaptOR achieved a bounding box Average Precision (AP) of 57.3% on MVOR+, nearly doubling the 29.6% achieved by standard non-adapted models.
Data Efficiency
In semi-supervised tests on the COCO dataset, AdaptOR achieved performance with only 1% of labeled data that was comparable to models trained on the full dataset, highlighting its efficiency in low-label regimes.
Qualitative Performance
Visual analysis confirms that the model accurately identifies clinicians even under heavy occlusion by surgical equipment or other personnel, maintaining consistent tracking across varying resolutions.
Conclusion
AdaptOR effectively bridges the gap between general-purpose vision models and the specialized requirements of the operating room. By leveraging geometric constraints and disentangled normalization, it enables accurate clinician tracking in privacy-sensitive, low-resolution environments without requiring manual OR labels. Future research will explore test-time adaptation and the integration of temporal video data to further enhance tracking stability.
Summary updated