Abstract
In this paper we introduce OperA, a transformer-based model that accurately predicts surgical phases from long video sequences. A novel attention regularization loss encourages the model to focus on high-quality frames during training. Moreover, the attention weights are utilized to identify characteristic high attention frames for each surgical phase, which could further be used for surgery summarization. OperA is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos, outperforming various state-of-the-art temporal refinement approaches.
Introduction
Surgical Phase Recognition is a critical component for the "Operating Room of the Future," enabling AI systems to provide real-time guidance, safety alerts, and automated procedure summarization. However, laparoscopic video data is inherently challenging due to its length and visual "noise"—including camera movement, smoke from cauterization, and blood obscuring the field of view.
Traditional sequential models like LSTMs often struggle with long-range dependencies in 30-60 minute procedures, while Temporal Convolutional Networks (TCNs) have limited receptive fields. OperA (Attention-Regularized Transformers) addresses these challenges by leveraging the global context of Transformers while introducing a novel mechanism to filter out uninformative visual data.
Methodology
OperA utilizes a transformer-based architecture designed for online sequential modeling. The pipeline consists of three main stages:
- Feature Extraction: A ResNet-50 backbone, pre-trained on surgical tasks, extracts visual features from each frame.
- Causal Transformer Stack: The core architecture comprises 11 self-attention layers. To enable real-time prediction, the model employs causal masking, ensuring that predictions at any time $t$ only depend on current and past frames.
- Attention Regularization ($L_{reg}$): This novel loss function penalizes the model for assigning high attention weights to frames where the CNN backbone exhibits high error or low confidence. By tying attention to the backbone's performance, the model learns to prioritize high-quality, informative frames over noisy ones.
Experimental Setup and Results
The model was evaluated on two laparoscopic cholecystectomy datasets: Cholec80 (80 videos, 7 phases) and MitiSW (85 videos, 8 phases). OperA was compared against state-of-the-art baselines including TeCNO and ResLSTM.
As shown in the results, OperA achieved a state-of-the-art accuracy of 91.26% on Cholec80, a significant improvement over previous temporal refinement methods. Ablation studies confirmed that the attention regularization term consistently improved both accuracy and F1-scores across both datasets.
Discussion: Attention as a Filter
A key insight from this work is the dual utility of the attention mechanism. Beyond improving classification accuracy, the attention weights serve as a natural filter for video quality. Frames with the Highest Attention (HA) typically contain clear, characteristic views of surgical actions (e.g., clipping a vessel), while Lowest Attention (LA) frames often correspond to obstructed views or motion blur. This property allows OperA to be used for automatic surgical summarization by selecting "highlight" frames based on attention scores.
Conclusion and Future Work
OperA demonstrates that attention-regularized transformers are highly effective for modeling long-term dependencies in surgical workflows while remaining robust to visual noise. Future work will explore scaling the model beyond 11 layers—currently limited by hardware constraints—and testing its generalization across a wider variety of laparoscopic procedures.
Summary updated