TL;DR
TopoOR introduces a novel way to model the complex, multi-actor environment of a surgical operating room (OR) using higher-order topological structures instead of simple graphs. By treating surgical interactions as "combinatorial complexes," the framework captures group dynamics and multimodal data (like 3D motion, audio, and robot logs) more effectively, significantly outperforming traditional models and LLMs in safety-critical tasks like sterility breach detection and workflow prediction.
What Problem Does This Paper Address?
The field of Surgical Data Science (SDS) aims to make surgeries safer and more efficient through computational modeling. Currently, researchers use Surgical Scene Graphs (SSGs) to map out the relationships between people, tools, and equipment.
However, the authors identify a major flaw: existing SSGs are "strictly dyadic." They only model pairwise relationships (e.g., Surgeon-Robot or Saw-Patient). In reality, surgical procedures are irreducibly polyadic—meaning they involve groups of entities working in a unified loop. For example, a surgeon guiding a robotic arm while watching a monitor and resecting a patient's bone is one single, complex interaction.
Standard graphs fragment this loop into isolated pairs, losing the joint spatial and kinematic constraints. Furthermore, current methods often "flatten" diverse data types (3D geometry, audio, text) into a single representation, which strips away the unique geometric structure essential for safety-critical reasoning.
Methods
The authors propose TopoOR, a framework rooted in algebraic topology.
- Combinatorial Complexes (CC): Instead of a standard graph, TopoOR models the OR as a CC.
- Rank-0 cells: Atomic entities like human joints, 3D objects, and "evidence nodes" (audio features or robot logs).
- Rank-1 cells: Interactions, such as human skeletal connections or spatial proximity between a nurse and a tool table.
- Rank-2 cells: Higher-order behaviors, such as a "functional hypercell" that encapsulates a multi-actor event (e.g., the surgeon-robot-patient interaction).
- Higher-Order Attention Network (HAT): This is the "brain" of the model. Unlike standard Graph Attention Networks (GAT) that only look at neighbors, HAT passes messages across the hierarchical structure of the complex. It uses a "rank-pair bias" to help the model distinguish between entity-level features (human kinematics) and aggregated group behaviors.
- Multimodal Integration: The system uses specialized "frozen" encoders to process different data: 3D pose estimation for humans, CNNs for monitor screens, and Large Language Models (LLMs) to process textual robot logs. These are all "anchored" into the 3D topological structure.
Datasets
The model was evaluated on the MM-OR dataset, a large-scale, multimodal operating room dataset. The authors tested TopoOR on three primary tasks:
- Sterility Breach Detection: Identifying when a non-sterile person or object gets too close to a sterile zone.
- Robot Phase Prediction: Identifying the current stage of a robotic procedure.
- Next-Action Anticipation: Predicting what the surgical team will do next.
Key Insights and Ideas
- Topological "Lifting": By "lifting" interactions into higher-order cells, the model natively preserves the manifold geometry of the OR. It doesn't just know that two things are related; it understands the structure of the group interaction.
- Subsuming Traditional Graphs: The authors prove that their topological representation is "strictly more expressive" than traditional scene graphs. In fact, TopoOR can be used to generate traditional scene graphs more accurately than models designed specifically for that task.
- Efficiency Matters: Despite its mathematical complexity, TopoOR is much lighter than modern Vision-Language Models (VLMs). It uses only 12 million parameters compared to the 7 billion parameters in the baseline LLM (MM2SG), making it much more suitable for real-time use in a live surgery.
Key Results
TopoOR demonstrated superior performance across the board compared to LLM-based and standard graph-based baselines:
- Sterility Breach Detection: Achieved 76.83% F1-score, significantly outperforming the LLM-based MM2SG (55.00%).
- Robot Phase Prediction: Reached a state-of-the-art 73.53% F1-score, compared to 56.90% for the LLM baseline.
- Next-Action Anticipation: Outperformed all baselines with a 41.10% F1-score.
- Inference Speed: TopoOR processed frames in roughly 59ms, whereas the quantized 7B parameter LLM took 194ms. This 3x speed advantage is crucial for intra-operative safety systems.
Limitations and Future Work
The authors acknowledge a few key limitations:
- Benchmark Scope: Current benchmarks focus on classification and regression. While TopoOR excels at these, the authors note that these metrics don't fully capture "real-world clinical utility," such as reducing a surgical team's cognitive load.
- Actionable Metrics: Future work should focus on "clinically actionable" outputs, such as active risk mitigation and context-aware error prevention.
- Dependency on Perception: The system relies on "frozen" perception modules for 3D pose and object detection. If the initial 3D estimation is poor, the topological model's performance will likely suffer.
Summary updated