Coefficient of Variation Masking: A Volatility-Aware Strategy for EHR Foundation Models

Rajna Fani, Rafi Al Attrach, David Restrepo, Yugang Jia, Leo Anthony Celi, Peter Schüffler

Abstract

Masked autoencoders (MAEs) are increasingly applied to electronic health records (EHR) for learning general-purpose representations that support diverse clinical tasks. However, existing approaches typically rely on uniform random masking, implicitly assuming all features are equally predictable. In reality, laboratory tests exhibit substantial heterogeneity in volatility: some biomarkers (e.g., sodium) remain stable, while others (e.g., lactate) fluctuate considerably and are more difficult to model. Clinically, volatile biomarkers often signal acute pathophysiology and require more sophisticated modeling to capture their complex temporal patterns. We propose a volatility-aware pretraining strategy, Coefficient of Variation Masking (CV-Masking), that adaptively adjusts masking probabilities according to the intrinsic variability of each feature. Combined with a value-only masking objective aligned with clinical workflows, CV-Masking yields systematic improvements over random and variance-based strategies. Experiments on a large panel of laboratory tests show that CV-Masking enhances reconstruction, improves downstream predictive performance, and accelerates convergence, producing more robust and clinically meaningful EHR representations.

Introduction

Foundation models in the medical domain often leverage self-supervised learning, specifically Masked Autoencoding (MAE), to learn representations from Electronic Health Records (EHR). By masking portions of a patient's record and tasking the model with reconstructing the missing data, these models learn the complex underlying structures of human physiology.

A significant limitation of current EHR foundation models is the use of uniform random masking. This approach treats all clinical features as equally informative or predictable. However, medical data is inherently heterogeneous. Stable biomarkers, such as sodium, are tightly regulated and easy to predict, offering little learning signal. Conversely, volatile markers like lactate or glucose can fluctuate rapidly during acute illness. These volatile markers are harder to model but are often the most clinically significant. This paper introduces CV-Masking, a strategy that prioritizes these high-volatility features to improve model robustness and clinical utility.

Methods

The proposed approach consists of two primary innovations: Coefficient of Variation Masking and a Value-Only Masked Autoencoder architecture.

CV-Masking Strategy

The Coefficient of Variation ($CV = \sigma / \mu$) is used as a scale-invariant metric for feature volatility. Features are ranked by their CV, and a curriculum-inspired masking strategy is applied: features in the top 25% of volatility are assigned a high masking probability (0.8), while the remaining features are assigned a lower probability (0.2). This forces the model to dedicate more capacity to learning the complex temporal patterns of volatile biomarkers.

Value-Only Masked Autoencoder (VO-MAE)

To better align with clinical reality, the authors developed the VO-MAE. In a hospital setting, clinicians typically know which test was ordered and when; the unknown information is the result itself. Unlike standard MAEs that mask the entire data triplet (time, code, and value), VO-MAE keeps the time and test code visible, masking only the numeric value.

Architecture of the Value-Only Masked Autoencoder (VO-MAE)

Dataset and Implementation

The models were trained and evaluated using the MIMIC-IV v3.1 database, focusing on 100 high-frequency laboratory tests across 40,000 ICU admissions. Data was standardized using the Medical Event Data Standard (MEDS).

Results

CV-Masking demonstrated significant improvements across reconstruction accuracy, clinical prediction, and training efficiency.

Reconstruction and Convergence

The CV-Masking strategy improved reconstruction accuracy for 71% of the laboratory tests compared to random masking. Notably, the model achieved peak performance in just 33 epochs, representing a 50% reduction in training time compared to random masking (67 epochs) and a 67% reduction compared to variance-based masking (100 epochs).

Downstream Clinical Performance

When evaluated on downstream clinical tasks using linear probing on frozen representations, the CV-masked model consistently outperformed both random and variance-based masking strategies across multiple metrics.

Downstream predictive performance across clinical tasks

The model also showed a 2.1x increase in sensitivity to changes in patient history during perturbation studies, suggesting that CV-Masking encourages the model to learn patient-specific temporal trajectories rather than relying on population-level averages.

Discussion and Conclusion

CV-Masking effectively addresses the heterogeneity of EHR data by focusing learning on volatile, clinically rich biomarkers. By aligning the masking objective with clinical workflows through VO-MAE, the resulting representations are more robust and predictive of critical outcomes like mortality and readmission.

Future work includes validating these findings on multi-center datasets to ensure generalizability across different patient populations and hospital systems. Additionally, exploring dynamic masking thresholds and extending the volatility-aware strategy to other data modalities, such as high-frequency vital signs, remains a promising direction for EHR foundation model research.

Summary updated

Abstract

Masked autoencoders (MAEs) are increasingly applied to electronic health records (EHR) for learning general-purpose representations that support diverse clinical tasks. However, existing approaches typically rely on uniform random masking, implicitly assuming all features are equally predictable. In reality, laboratory tests exhibit substantial heterogeneity in volatility: some biomarkers (e.g., sodium) remain stable, while others (e.g., lactate) fluctuate considerably and are more difficult to model. Clinically, volatile biomarkers often signal acute pathophysiology and require more sophisticated modeling to capture their complex temporal patterns. We propose a volatility-aware pretraining strategy, Coefficient of Variation Masking (CV-Masking), that adaptively adjusts masking probabilities according to the intrinsic variability of each feature. Combined with a value-only masking objective aligned with clinical workflows, CV-Masking yields systematic improvements over random and variance-based strategies. Experiments on a large panel of laboratory tests show that CV-Masking enhances reconstruction, improves downstream predictive performance, and accelerates convergence, producing more robust and clinically meaningful EHR representations.
BibTeX
@misc{fani2025coefficient,
  author = {Rajna Fani and Rafi Al Attrach and David Restrepo and Yugang Jia and Leo Anthony Celi and Peter Schüffler},
  title = {{Coefficient of Variation Masking: A Volatility-Aware Strategy for EHR Foundation Models}},
  year = {2025},
  eprint = {2512.05216},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.05216},
}
Figure