In-Situ Visualization for Surgical Navigation

This direction is merged into another direction: 2D-to-3D cognitive mapping. Its last version stays readable and citable here.

Version 1 of 139 papers51 conceptsfrozen 2026-10-04

In traditional surgery, a practitioner must perform a complex mental feat: looking at a two-dimensional monitor—displaying a CT scan, MRI, or ultrasound—and "projecting" that information into the three-dimensional body of the patient in front of them. This process, known as mental mapping, creates a significant cognitive load and a "visuo-motor disconnect," where the surgeon’s hands work in one location while their eyes are fixed on a remote screen.

A growing body of research is moving toward in-situ visualization, a paradigm where medical data is overlaid directly onto the patient’s anatomy using Augmented Reality (AR) and Mixed Reality (MR). By aligning digital guidance with the physical workspace, these systems offload the task of spatial integration from the human brain to the software interface.

The Cognitive Bridge

The primary hurdle in modern image-guided surgery is the "split-attention effect." As Marco Lai and Caifeng Shan observe, surgeons frequently have to look away from the operative site to consult external monitors. This diversion not only breaks focus but forces the surgeon to maintain a mental spatial map that is prone to error.

In-situ visualization solves this by creating what Christoph Bichlmeier and Sandro Michael Heining describe as "X-ray vision." By superimposing 3D anatomical data directly onto the patient’s body, the system eliminates the need for mental translation. Lukas Jud and Mazda Farshad argue that this mechanism reduces cognitive load by aligning digital guidance—such as planned trajectories or 3D models—with the physical environment, allowing the surgeon to "see through" tissue to critical structures like arteries and nerves.

Offloading Spatial Integration

The shift from 2D screens to 3D overlays is more than an ergonomic improvement; it is a fundamental change in how information is processed. Anas Amin Preukschas and Martin Wagner 0001 demonstrate that 3D and VR visualizations significantly outperform traditional 2D scans in decision-making tasks because they provide "cognitive load offloading." The software handles the spatial reconstruction, leaving the practitioner free to focus on the surgical maneuver itself.

This is particularly evident in needle-based procedures. Tabea Borde and Ming Li show that holographic projections of needle trajectories allow clinicians to maintain a continuous focus on the patient. By centering the target on the device's axis, complex 3D paths are simplified into 2D alignment tasks, further reducing the mental effort required for orientation.

Bridging Planning and Execution

One of the most difficult aspects of surgery is ensuring that the preoperative plan—meticulously designed on a computer—is executed precisely on the operating table. Letian Ai and Alejandro Martin-Gomez describe this as "on-the-fly" planning, where Mixed Reality allows surgeons to adjust tool paths directly on the patient's anatomy.

This direct mapping has shown measurable improvements in precision across various fields:

  • Orthopedics: Florentin Liebmann and Marco von Atzigen developed a standalone AR system for spinal surgery that provides radiation-free guidance for pedicle screw placement.
  • Soft Tissue: Siyi Chen and Steve Guofang Shen used volumetric overlays for facial fat grafting, reducing reconstruction errors by 35%.
  • Trauma: Hoijoon Jung and Ge Jin demonstrated that holographic maps of rib fractures allow for smaller incisions and faster localization of injuries hidden behind dense bone.

Democratizing Surgical Expertise

Perhaps the most compelling evidence for the power of in-situ visualization is its impact on novice practitioners. Lei Gao and Yidi Xu found that Mixed Reality guidance allowed residents to perform spinal punctures with significantly fewer attempts than traditional landmark-guided methods. The real-time anatomical overlay effectively compensates for a lack of clinical experience by making the internal structures "visible" and the optimal trajectory intuitive.

Beyond the operating room, this spatial grounding of data is being used to improve patient communication and clinical handovers. Dmitri Presnov and Andreas Kolb have explored mapping abstract clinical data—like pain intensity or symptoms—directly onto 3D human avatars. This transforms text-heavy medical records into a "virtual patient" interface, leveraging our natural anatomical familiarity to speed up information transfer.

The Future of the "Virtual Monitor"

As hardware matures, the field is moving away from bulky, fixed equipment toward wearable, egocentric interfaces. Long Qian and Nassir Navab suggest that "virtual monitors" streamed to head-mounted displays will eventually replace the wall-mounted screens of today. By locking digital information directly onto the patient's anatomy, these systems allow surgeons to maintain a continuous, ergonomic focus on the task at hand, turning the patient's own body into the primary interface for the surgery.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_in_situ_visualization_for_surgical_navig_9eb9113a,
  title = {In-Situ Visualization for Surgical Navigation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/in-situ-visualization-for-surgical-navigation/rdr_9eb9113a96f14206a558affb1d1217c6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_in_situ_visualization_for_surgical_navig_9eb9113a_v1,
  title = {In-Situ Visualization for Surgical Navigation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/in-situ-visualization-for-surgical-navigation/rdr_9eb9113a96f14206a558affb1d1217c6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}