When a clinician reviews a patient’s medical history, they do not see a flat list of disconnected events. Instead, they naturally navigate a deeply nested hierarchy: individual medical codes (such as a specific lab result or drug dosage) are grouped into discrete clinical encounters (visits), which are then organized chronologically to form a multi-visit patient journey. Within each visit, there is an underlying clinical logic—a diagnosis directly prompts a specific treatment or lab test.
Traditional deep learning models often ignore this structure. By treating Electronic Health Records (EHR) as flat sequences or "bags of features," standard architectures discard the vital relational context that links observations to actions. Hierarchical representation learning addresses this flaw by designing neural architectures that mirror the nested, multi-level nature of clinical data.
The Flaw of Flattening Clinical Data
Why does flattening data hurt model performance? Consider a patient with a complex, years-long medical history. If an algorithm processes their record as a single, continuous sequence of codes, two major issues arise:
- Context Dilution: A specific diagnosis and its corresponding treatment might be separated by dozens of unrelated codes from other visits. In a flat sequence model, the direct causal link between that specific observation and action becomes obscured.
- Temporal Imbalance: Long-term historical trends can easily wash out immediate, acute clinical context. As noted by Liu et al. in their development of SHAPE, decoupling immediate clinical events from long-term longitudinal patterns is essential for ensuring that a model accurately grasps a patient's current state without being overwhelmed by historical data.
When models treat clinical data as flat sequences, they struggle to capture the relative importance of primary versus secondary diagnoses within a single encounter, a limitation highlighted by Rasmy et al. in the context of Med-BERT. By contrast, structuring models to respect the natural hierarchy of medical data provides a powerful inductive bias, allowing the network to preserve the logical relationships inherent in clinical decision-making.
Mechanisms for Multi-Level Representation
To capture these nested dependencies, researchers have developed several distinct architectural strategies that operate at multiple granularities simultaneously.
Multi-Level Attention Networks
Instead of applying a single attention mechanism across an entire history, hierarchical models apply attention sequentially at different levels of the data tree. For example, Fang et al. proposed the Interpretable Hierarchical Attention Network (IHAN), which utilizes a three-level attention mechanism operating across codes, encounters, and data types. This allows the model to learn the relative importance of a specific prescription versus a lab result within a single visit, while simultaneously evaluating the significance of that entire visit within the patient's multi-year history. A similar multi-level self-attention strategy is pursued by Peng et al. to map out complex patient journeys.
Graph and Hypergraph Transformers
Clinical hierarchies are not always strictly linear; they can also be represented as graphical structures where nodes represent medical concepts and edges represent clinical dependencies. Choi et al. developed the Graph Convolutional Transformer (GCT), which uses data statistics to guide a Transformer's attention mechanism to learn the hidden hierarchical and causal relationships between diagnoses, treatments, and lab tests.
To capture even more complex, multi-dimensional relationships, Chen et al. introduced TRANS, which treats EHR data as a temporal heterogeneous graph to balance long-term trends with immediate clinical context.
Hierarchical Contrastive Learning
When labeled clinical data is scarce, self-supervised learning can leverage the data's inherent structure as a supervisory signal. In COMET, Wang et al. demonstrated a hierarchical contrastive framework for medical time-series that enforces consistency across four nested levels: individual observations, segments, trials, and unique patient identities. This ensures that fine-grained temporal details are successfully integrated into broader clinical contexts without losing valuable metadata.
Extending Hierarchy to Unstructured Text and Vision
The necessity of hierarchical modeling extends beyond structured medical codes to unstructured clinical narratives and medical imaging.
Clinical notes possess an explicit linguistic hierarchy: words form sentences, sentences form local text segments, and segments form complete notes that document a patient's longitudinal history. Zhang et al. demonstrated in FTL-Trans that preserving the relationship between local text segments and parent documents prevents the loss of high-level context, significantly improving prediction accuracy over flattened sequence models. Similarly, Kim et al. utilized multi-level hypergraphs in Clinical Note Owns its Hierarchy to balance these granularities.
Even in medical vision tasks, such as histopathology, data is inherently nested. Malik et al. showed that treating data samples as independent entities leaves models vulnerable to adversarial attacks. By leveraging the patient-slide-patch relationship during training, their framework builds more resilient representations that reflect the true multi-level structure of the specialized domain.
Why the Idea Works
Hierarchical EHR representation works because it aligns the architecture's mathematical constraints with the clinical reality of how medical data is generated.
By grouping codes into discrete events before aggregating them over time—as validated by Lee et al.—the model is constrained to learn representations that respect clinical logic. It prevents the "forgetting" behavior common in long, flat sequence models (a problem addressed by the bidirectional architecture of Long et al.), while ensuring that the model can simultaneously perform short-term bedside monitoring and long-term health risk assessment. Ultimately, mirroring the clinician's structured decision-making process provides the optimal inductive bias for healthcare predictive modeling.
Key Foundational Work
Using an Ensemble of Linear and Deep Learning Models in the SMM4H 2017 Medical Concept Normalisation Task (2017) — contributed 1 concept elements to this direction.
Statistical Relational Learning to Predict Primary Myocardial Infarction from Electronic Health Records (2012) — contributed 1 concept elements to this direction.