UniHPF : Universal Healthcare Predictive Framework with Zero Domain Knowledge

Kyunghoon Hur, Jungwoo Oh, Junu Kim, Jiyoun Kim, Min Jae Lee, Eunbyeol Cho, Seong-Eun Moon, Young-Hak Kim, Edward Choi

TL;DR

UniHPF (Universal Healthcare Predictive Framework) is a new approach to medical forecasting that eliminates the need for expensive, expert-driven data preprocessing. By treating all Electronic Health Record (EHR) data—including medical codes and database schemas—as raw text and processing it through a hierarchical Transformer, the framework enables models to learn effectively across different hospital systems and pooled datasets without any specialized medical domain knowledge.

What Problem Does This Paper Address?

Building predictive models for healthcare is notoriously difficult due to data heterogeneity. Every hospital has its own way of storing data: they use different database schemas and different medical coding standards (like ICD-9 vs. ICD-10).

Currently, the standard solution is to have medical experts manually "clean" the data, selecting specific features and mapping codes to a Common Data Model (CDM). This process is slow, expensive, and doesn't scale. Furthermore, it makes "multi-source learning"—where you train a model on data from multiple hospitals—nearly impossible because the "vocabulary" of one hospital's EHR system often doesn't overlap with another's.

A typical EHR predictive model framework involves domain-knowledge-based pre-processing for each medical center’s schema.

As shown in Figure 4, the traditional pipeline is siloed and requires significant manual intervention for every new system.

Methods

UniHPF shifts the paradigm by treating EHR data as a sequence of textual events rather than a collection of discrete codes.

  1. Text-based Embedding: Instead of assigning a unique ID to every medical code, UniHPF converts codes into their textual descriptions (e.g., "401.9" becomes "unspecified essential hypertension").
  2. Schema-Agnostic Input: The framework uses all available information in a medical event: the event type (e.g., "prescription"), the feature name (e.g., "drug name"), and the feature value (e.g., "vancomycin").
  3. Hierarchical Architecture: The model uses a two-stage Transformer approach. An event encoder ($f$) first processes the sub-word tokens of a single medical event into an embedding. Then, an event aggregator ($g$) processes the sequence of these event embeddings to make a final prediction.
  4. Handling Numbers: To deal with continuous values (like lab results), UniHPF uses "digit-place embedding," splitting numbers digit-by-digit so the model can learn the magnitude of values without manual quantization.
Overview of UniHPF architecture showing the conversion of medical events into text tokens and then into embeddings.

Datasets

The authors evaluated UniHPF using three major open-source ICU datasets:

  • MIMIC-III
  • MIMIC-IV
  • eICU

These datasets are ideal for testing because while MIMIC-III and IV share some similarities, eICU comes from multiple different hospitals and uses a completely distinct coding system and schema.

Key Insights and Ideas

  • "More is More": One of the most surprising findings is that using all available features (schema-agnostic) often outperforms models where experts hand-picked the "relevant" features. The model is capable of filtering out the noise itself.
  • Text as a Universal Translator: By converting codes to descriptions, the model can bridge the gap between hospitals. A "vancomycin" prescription looks the same to the model regardless of whether the underlying database calls it "ITEMID 123" or "DRUG_CODE A-B-C."
  • Scaling through Pooling: In conventional models, training on data from multiple hospitals often decreases performance because the vocabulary of unique codes explodes. In UniHPF, pooling data from different sources consistently improves performance because the underlying medical concepts (expressed in text) are shared across datasets.

Key Results

UniHPF was tested on seven tasks, including mortality prediction, length of stay, and diagnosis.

  • Single Domain: UniHPF generally outperformed or matched expert-engineered benchmarks across all three datasets.
  • Multi-Source Learning: As shown in Figure 3, when training on a pooled dataset (MIMIC-III + MIMIC-IV + eICU), UniHPF showed significant performance gains. In contrast, conventional embedding models (like SAnD or Rajkomar) often saw their performance drop when datasets were combined.
  • Transfer Learning: The model demonstrated strong "zero-shot" capabilities, performing well on a new hospital's data even before being fine-tuned on it.
Pooled learning results showing that UniHPF (column D) consistently improves when more data sources are added.

Limitations and Future Work

  • Pre-training Challenges: The authors experimented with standard NLP pre-training (like Masked Language Modeling), but it didn't provide a significant boost. They suggest that a pre-training strategy specifically designed for the temporal and structural nature of EHR data is still needed.
  • Computational Overhead: Processing every single feature as text is computationally expensive. UniHPF requires significantly more VRAM than simpler, flattened models (using nearly 80GB per GPU in some tests).
  • Language Dependency: The current framework assumes that the EHR descriptions are in the same language (English). Extending this to multi-lingual EHR systems remains an open challenge.

Summary updated

Abstract

Despite the abundance of Electronic Healthcare Records (EHR), its heterogeneity restricts the utilization of medical data in building predictive models. To address this challenge, we propose Universal Healthcare Predictive Framework (UniHPF), which requires no medical domain knowledge and minimal pre-processing for multiple prediction tasks. Experimental results demonstrate that UniHPF is capable of building large-scale EHR models that can process any form of medical data from distinct EHR systems. We believe that our findings can provide helpful insights for further research on the multi-source learning of EHRs.
BibTeX
@misc{hur2022unihpf,
  author = {Kyunghoon Hur and Jungwoo Oh and Junu Kim and Jiyoun Kim and Min Jae Lee and Eunbyeol Cho and Seong-Eun Moon and Young-Hak Kim and Edward Choi},
  title = {{UniHPF : Universal Healthcare Predictive Framework with Zero Domain Knowledge}},
  year = {2022},
  eprint = {2207.09858},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2207.09858},
}
Figure