Integrated Visualization of Patient Data

Dmitri Presnov, Julia Kurz, Judith Willkomm, Johannes Dillmann, Daniel Remmel, Robert Zilke, Veit Braun, Cornelius Schubert, Andreas Kolb

Abstract

Electronic health records (EHR) systematically represent patient data in digital form. However, text and visualization-based EHR systems are often poorly integrated into hospital workflows due to their complex and non-intuitive access structures. This is especially disadvantageous in clinical cooperative situations, such as neurosurgical wards, which require efficient, task-specific information transfer. This paper introduces a novel concept of anatomically integrated in-place visualization designed to support cooperative tasks. By using a virtual patient's body as a spatial representation of visually encoded abstract medical data, we propose an approach that aligns with the visual traditions of medicine. We provide a set of formal requirements for these in-place visualizations, apply them to neurological symptoms related to spinal disc herniation, and present a mobile prototype evaluated by 10 neurosurgeons.

Introduction

In a high-stakes neurosurgical ward, the speed and accuracy of information transfer during shift handovers or ward rounds are critical. Despite the digital transition, current Electronic Health Records (EHRs) often hinder rather than help this process. They are frequently text-heavy and buried under layers of menus, leading to a "loss of overview." To address this, we propose moving away from abstract charts toward anatomically integrated in-place visualization—a system where patient data is mapped directly onto a 3D virtual human avatar.

Design Goals and Technical Approach

The core philosophy of this system is to tap into the existing mental models of clinicians. Doctors are trained to think anatomically; therefore, placing data directly on a body model reduces the cognitive load required to translate text into a clinical picture.

Anatomical Mapping and Visual Encoding

To represent neurological status, the system utilizes a 3D human avatar with refined geometry. Specifically, we mapped "dermatomes"—areas of skin supplied by specific spinal nerves—using indexed texture maps. Abstract medical data is then mapped to specific visual variables:

  • Hue: Represents the category of the symptom (e.g., Red for pain, Purple for muscle weakness/paresis).
  • Saturation and Brightness: Represent the intensity or severity of the symptom.
  • 3D Solid Textures: To represent secondary properties, such as whether pain is constant or triggered by movement, we employed procedural 3D solid textures (Perlin noise). This ensures visual consistency regardless of the underlying mesh complexity.

Overcoming Visibility Challenges

Clinical data often resides "inside" the body. To manage this, the system uses dynamic, view-dependent semi-transparency to reveal hidden structures without losing the anatomical context of the skin or muscle layers. Furthermore, for anatomical features too small to render effectively in 3D (like specific tendons), we introduced "proxies"—scaled 2D icons projected onto the skin surface to ensure these vital data points remain visible.

Implementation and Evaluation

The prototype was developed using the Vulkan API on Android to ensure high-performance rendering on mobile devices, facilitating ubiquitous access during rounds. The system was evaluated using seven real-life clinical case descriptions of spinal disc herniation.

Key Results

Ten neurosurgeons participated in the evaluation, yielding the following insights:

  • Intuitive Interpretation: In spontaneous interpretation tasks with no prior training, all 10 surgeons correctly identified muscle weakness, and 8 out of 10 correctly identified radicular pain based solely on the 3D visualization.
  • Temporal Tracking: Using a timeline slider, all participants correctly interpreted the "healing progress" (improvement or deterioration) of a patient over time without requiring a legend.
  • Efficiency: One participant noted that the system felt as intuitive as reviewing an MRI scan, allowing them to "see" the patient's status immediately rather than reading through notes.

Discussion and Future Work

The concept of "Injectivity"—ensuring every unique medical property has a unique visual representation—proved vital for clarity. However, the system currently faces limitations regarding non-anatomical data, such as blood work or laboratory results, which do not have a natural physical location on the body.

Future iterations will focus on managing visual clutter as more symptoms are added, potentially through advanced filtering and Level of Detail (LOD) management. While the current prototype focuses on spinal disc herniation, the framework is designed to be scalable to other clinical contexts, eventually integrating a broader range of EHR data into the 3D anatomical interface.

Summary updated

Abstract

Electronic health records (EHR) systematically represent patient data in digital form. However, text and visualization based EHR systems are poorly integrated in the hospital workflow due to their complex and rather non-intuitive access structure. This is especially disadvantageous in clinical cooperative situations that require an efficient, task specific information transfer. In this paper we introduce a novel concept of anatomically integrated in-place visualization designed to engage with cooperative tasks on a neurosurgical ward. Based on the findings of our field studies and the derived design goals, we propose an approach that follows a visual tradition in medicine, which is tightly related with anatomy, by using a virtual patient's body as spatial representation of visually encoded abstract medical data. More specifically, we provide a generic set of formal requirements for these kinds of in-place visualizations, we apply these requirements in order to achieve a specific visualization of neurological symptoms related to the differential diagnosis of spinal disc herniation, and we present a prototypical implementation of the visualization concept on a mobile device. Moreover, we discuss various challenges related to visual encoding and visibility of the body model components. Finally, the prototype is evaluated by 10 neurosurgeons, who assess the validity and the further potential of the proposed approach.
BibTeX
@misc{presnov2019integrated,
  author = {Dmitri Presnov and Julia Kurz and Judith Willkomm and Johannes Dillmann and Daniel Remmel and Robert Zilke and Veit Braun and Cornelius Schubert and Andreas Kolb},
  title = {{Integrated Visualization of Patient Data}},
  year = {2019},
  eprint = {1906.07720},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.07720},
}
Figure