Abstract
Electronic health records (EHR) systematically represent patient data in digital form. However, text and visualization-based EHR systems are often poorly integrated into hospital workflows due to their complex and non-intuitive access structures. This is especially disadvantageous in clinical cooperative situations, such as neurosurgical wards, which require efficient, task-specific information transfer. This paper introduces a novel concept of anatomically integrated in-place visualization designed to support cooperative tasks. By using a virtual patient's body as a spatial representation of visually encoded abstract medical data, we propose an approach that aligns with the visual traditions of medicine. We provide a set of formal requirements for these in-place visualizations, apply them to neurological symptoms related to spinal disc herniation, and present a mobile prototype evaluated by 10 neurosurgeons.
Introduction
In a high-stakes neurosurgical ward, the speed and accuracy of information transfer during shift handovers or ward rounds are critical. Despite the digital transition, current Electronic Health Records (EHRs) often hinder rather than help this process. They are frequently text-heavy and buried under layers of menus, leading to a "loss of overview." To address this, we propose moving away from abstract charts toward anatomically integrated in-place visualization—a system where patient data is mapped directly onto a 3D virtual human avatar.
Design Goals and Technical Approach
The core philosophy of this system is to tap into the existing mental models of clinicians. Doctors are trained to think anatomically; therefore, placing data directly on a body model reduces the cognitive load required to translate text into a clinical picture.
Anatomical Mapping and Visual Encoding
To represent neurological status, the system utilizes a 3D human avatar with refined geometry. Specifically, we mapped "dermatomes"—areas of skin supplied by specific spinal nerves—using indexed texture maps. Abstract medical data is then mapped to specific visual variables:
- Hue: Represents the category of the symptom (e.g., Red for pain, Purple for muscle weakness/paresis).
- Saturation and Brightness: Represent the intensity or severity of the symptom.
- 3D Solid Textures: To represent secondary properties, such as whether pain is constant or triggered by movement, we employed procedural 3D solid textures (Perlin noise). This ensures visual consistency regardless of the underlying mesh complexity.
Overcoming Visibility Challenges
Clinical data often resides "inside" the body. To manage this, the system uses dynamic, view-dependent semi-transparency to reveal hidden structures without losing the anatomical context of the skin or muscle layers. Furthermore, for anatomical features too small to render effectively in 3D (like specific tendons), we introduced "proxies"—scaled 2D icons projected onto the skin surface to ensure these vital data points remain visible.
Implementation and Evaluation
The prototype was developed using the Vulkan API on Android to ensure high-performance rendering on mobile devices, facilitating ubiquitous access during rounds. The system was evaluated using seven real-life clinical case descriptions of spinal disc herniation.
Key Results
Ten neurosurgeons participated in the evaluation, yielding the following insights:
- Intuitive Interpretation: In spontaneous interpretation tasks with no prior training, all 10 surgeons correctly identified muscle weakness, and 8 out of 10 correctly identified radicular pain based solely on the 3D visualization.
- Temporal Tracking: Using a timeline slider, all participants correctly interpreted the "healing progress" (improvement or deterioration) of a patient over time without requiring a legend.
- Efficiency: One participant noted that the system felt as intuitive as reviewing an MRI scan, allowing them to "see" the patient's status immediately rather than reading through notes.
Discussion and Future Work
The concept of "Injectivity"—ensuring every unique medical property has a unique visual representation—proved vital for clarity. However, the system currently faces limitations regarding non-anatomical data, such as blood work or laboratory results, which do not have a natural physical location on the body.
Future iterations will focus on managing visual clutter as more symptoms are added, potentially through advanced filtering and Level of Detail (LOD) management. While the current prototype focuses on spinal disc herniation, the framework is designed to be scalable to other clinical contexts, eventually integrating a broader range of EHR data into the 3D anatomical interface.
Summary updated