TopoOR: A Unified Topological Scene Representation for the Operating Room

Tony Danjun Wang, Ka Young Kim, Tolga Birdal, Nassir Navab, Lennart Bastian

TL;DR

TopoOR introduces a novel way to model the complex, multi-actor environment of a surgical operating room (OR) using higher-order topological structures instead of simple graphs. By treating surgical interactions as "combinatorial complexes," the framework captures group dynamics and multimodal data (like 3D motion, audio, and robot logs) more effectively, significantly outperforming traditional models and LLMs in safety-critical tasks like sterility breach detection and workflow prediction.

What Problem Does This Paper Address?

The field of Surgical Data Science (SDS) aims to make surgeries safer and more efficient through computational modeling. Currently, researchers use Surgical Scene Graphs (SSGs) to map out the relationships between people, tools, and equipment.

However, the authors identify a major flaw: existing SSGs are "strictly dyadic." They only model pairwise relationships (e.g., Surgeon-Robot or Saw-Patient). In reality, surgical procedures are irreducibly polyadic—meaning they involve groups of entities working in a unified loop. For example, a surgeon guiding a robotic arm while watching a monitor and resecting a patient's bone is one single, complex interaction.

Standard graphs fragment this loop into isolated pairs, losing the joint spatial and kinematic constraints. Furthermore, current methods often "flatten" diverse data types (3D geometry, audio, text) into a single representation, which strips away the unique geometric structure essential for safety-critical reasoning.

Methods

The authors propose TopoOR, a framework rooted in algebraic topology.

  • Combinatorial Complexes (CC): Instead of a standard graph, TopoOR models the OR as a CC.
  • Rank-0 cells: Atomic entities like human joints, 3D objects, and "evidence nodes" (audio features or robot logs).
  • Rank-1 cells: Interactions, such as human skeletal connections or spatial proximity between a nurse and a tool table.
  • Rank-2 cells: Higher-order behaviors, such as a "functional hypercell" that encapsulates a multi-actor event (e.g., the surgeon-robot-patient interaction).
  • Higher-Order Attention Network (HAT): This is the "brain" of the model. Unlike standard Graph Attention Networks (GAT) that only look at neighbors, HAT passes messages across the hierarchical structure of the complex. It uses a "rank-pair bias" to help the model distinguish between entity-level features (human kinematics) and aggregated group behaviors.
  • Multimodal Integration: The system uses specialized "frozen" encoders to process different data: 3D pose estimation for humans, CNNs for monitor screens, and Large Language Models (LLMs) to process textual robot logs. These are all "anchored" into the 3D topological structure.

Datasets

The model was evaluated on the MM-OR dataset, a large-scale, multimodal operating room dataset. The authors tested TopoOR on three primary tasks:

  1. Sterility Breach Detection: Identifying when a non-sterile person or object gets too close to a sterile zone.
  2. Robot Phase Prediction: Identifying the current stage of a robotic procedure.
  3. Next-Action Anticipation: Predicting what the surgical team will do next.

Key Insights and Ideas

  • Topological "Lifting": By "lifting" interactions into higher-order cells, the model natively preserves the manifold geometry of the OR. It doesn't just know that two things are related; it understands the structure of the group interaction.
  • Subsuming Traditional Graphs: The authors prove that their topological representation is "strictly more expressive" than traditional scene graphs. In fact, TopoOR can be used to generate traditional scene graphs more accurately than models designed specifically for that task.
  • Efficiency Matters: Despite its mathematical complexity, TopoOR is much lighter than modern Vision-Language Models (VLMs). It uses only 12 million parameters compared to the 7 billion parameters in the baseline LLM (MM2SG), making it much more suitable for real-time use in a live surgery.

Key Results

TopoOR demonstrated superior performance across the board compared to LLM-based and standard graph-based baselines:

  • Sterility Breach Detection: Achieved 76.83% F1-score, significantly outperforming the LLM-based MM2SG (55.00%).
  • Robot Phase Prediction: Reached a state-of-the-art 73.53% F1-score, compared to 56.90% for the LLM baseline.
  • Next-Action Anticipation: Outperformed all baselines with a 41.10% F1-score.
  • Inference Speed: TopoOR processed frames in roughly 59ms, whereas the quantized 7B parameter LLM took 194ms. This 3x speed advantage is crucial for intra-operative safety systems.

Limitations and Future Work

The authors acknowledge a few key limitations:

  • Benchmark Scope: Current benchmarks focus on classification and regression. While TopoOR excels at these, the authors note that these metrics don't fully capture "real-world clinical utility," such as reducing a surgical team's cognitive load.
  • Actionable Metrics: Future work should focus on "clinically actionable" outputs, such as active risk mitigation and context-aware error prevention.
  • Dependency on Perception: The system relies on "frozen" perception modules for 3D pose and object detection. If the initial 3D estimation is poor, the topological model's performance will likely suffer.

Summary updated

Abstract

Surgical Scene Graphs abstract the complexity of surgical operating rooms (OR) into a structure of entities and their relations, but existing paradigms suffer from strictly dyadic structural limitations. Frameworks that predominantly rely on pairwise message passing or tokenized sequences flatten the manifold geometry inherent to relational structures and lose structure in the process. We introduce TopoOR, a new paradigm that models multimodal operating rooms as a higher-order structure, innately preserving pairwise and group relationships. By lifting interactions between entities into higher-order topological cells, TopoOR natively models complex dynamics and multimodality present in the OR. This topological representation subsumes traditional scene graphs, thereby offering strictly greater expressivity. We also propose a higher-order attention mechanism that explicitly preserves manifold structure and modality-specific features throughout hierarchical relational attention. In this way, we circumvent combining 3D geometry, audio, and robot kinematics into a single joint latent representation, preserving the precise multimodal structure required for safety-critical reasoning, unlike existing methods. Extensive experiments demonstrate that our approach outperforms traditional graph and LLM-based baselines across sterility breach detection, robot phase prediction, and next-action anticipation
BibTeX
@misc{wang2026topoor,
  author = {Tony Danjun Wang and Ka Young Kim and Tolga Birdal and Nassir Navab and Lennart Bastian},
  title = {{TopoOR: A Unified Topological Scene Representation for the Operating Room}},
  year = {2026},
  eprint = {2603.09466},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.09466},
}
Figure