Intensive-care notes are written in shorthand. A note may read only "RF worsening, OD?" Here "RF" could mean renal failure or respiratory failure, and "OD" could mean overdose or once daily. The words around the abbreviation rarely settle which sense is meant. Language models trained on general or biomedical text get this badly wrong. Embeddings trained on PubMed expand ICU abbreviations correctly only 28% of the time, because PubMed reads "OD" as optical density (Exploiting Task-Oriented Resources to Learn Word Embeddings for Clinical Abbreviation Expansion). The hardest cases are abbreviations and senses that never appear in the labeled training data, and these drive most remaining errors. Yet every ICU note comes with signals that the text lacks. It has a timestamp, it belongs to a patient whose vital signs are recorded hourly, and the same patient's other notes contain the same abbreviations a few hours earlier or later.
We propose to use those signals as a source of supervision rather than as extra input features. The core idea is a smoothness penalty over time on the model's sense predictions. Two mentions of the same abbreviation in the same patient's stay should receive the same sense when they are close in time and the patient's physiological state has not changed much between them. When the physiology jumps, for example when the patient is intubated or creatinine spikes, the penalty relaxes and the sense is free to change. Because this penalty needs no labels, it can be applied to every unlabeled mention in MIMIC-III. That includes mentions of abbreviations held out entirely from supervision, so the model learns about labelless senses by propagating evidence along each patient's timeline.
Core Claim
For abbreviation senses that have no labeled examples, a model trained with a physiology-gated temporal penalty on unlabeled notes will outperform two simpler alternatives. The first matches mentions against dictionary definitions alone. The second also concatenates current vital signs as features. The gain should come from the penalty linking mentions across time within a stay, not from the vitals as such. A second claim follows: the gate matters. A plain "one sense per stay" smoothing, with time adjacency but no physiology, will help on average but will hurt on the subset of stays where the correct sense actually changes. The physiology gate is what recovers that subset.
Where This Sits in Prior Work
Temporal smoothness penalties supply the method. Dynamic Word Embeddings for Evolving Semantic Discovery jointly factorizes yearly co-occurrence matrices with a penalty on the distance between consecutive years' embeddings. It warns that too strong a penalty over-dampens genuine sudden shifts. Tensor Decompositions for Temporal Knowledge Base Completion adds the same kind of penalty between consecutive timestamp embeddings and reports up to 0.07 MRR gains on ICEWS, but it only works for entities seen during training. Temporal graph-Laplacian penalties, which pull predictions at adjacent time points together, suppress jitter in motion segmentation (Temporal Subspace Clustering for Human Motion Segmentation) and denoise time series (Out-of-Sample Extension for Dimensionality Reduction of Noisy Time Series). Context-Aware Feature and Label Fusion for Facial Action Unit Intensity Estimation With Partially Labeled Data uses temporal smoothness to fill in missing labels. Temporally-Constrained Group Sparse Learning for Longitudinal Data Analysis adds fused smoothness on model weights across clinic visits.
All of these smooth latent coordinates or numeric outputs with a fixed strength. We differ in two ways. We smooth a categorical prediction, the sense distribution of a text mention. And the strength of the penalty is set by a second, non-textual signal, the patient's physiology, which directly answers the over-dampening concern raised in the dynamic word embeddings paper.
Cold-start methods supply the trick for handling unseen senses. Predict multi-type drug–drug interactions in cold start scenario regresses intrinsic attributes into a relational embedding space. Bridging chemical structure and conceptual knowledge enables accurate prediction of compound-protein interaction trains a network to predict knowledge-graph embeddings from molecular fingerprints. MOTIVE: A Drug-Target Interaction Graph For Inductive Link Prediction shows that intrinsic cell-imaging features can stand in for graph edges for unseen genes. We reuse their shared idea, mapping an intrinsic description into the scoring space, by embedding each sense from its dictionary definition. All three, however, treat the cold entity as static. The CSMDDI paper also shows that errors compound when nothing but the intrinsic mapping is available. Our temporal penalty is meant to supply a second, label-free signal on top of the definition mapping.
Work on sparse-context disambiguation both motivates and constrains the design. Wan et al. disambiguate short comments using their parent article. ERNIE: Enhanced Language Representation with Informative Entities injects knowledge-graph entity embeddings and raises FIGER strict accuracy from 52.04% to 57.19%. Both show that sparse mentions need context imported from outside the sentence. The key constraint comes from Efficient Biomedical Entity Linking: Clinical Text Standardization with Low-Resource Techniques. There, adding window or whole-document text context often hurt clinical linking, and only attention-based context helped. We take this as evidence against simply giving the model more text, and as motivation for a context source of a different kind.
The domain gap defines the setting. MeDAL: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretraining builds 14.3M pretraining examples by replacing full phrases with abbreviations, but it relies on a fixed dictionary. In the task-oriented-resources paper, 27% of errors are out-of-vocabulary terms and 54% are rare abbreviations with too few examples. Our method targets the 54%: abbreviations present in a dictionary but without enough labels. Truly out-of-vocabulary terms with no definition are out of scope, and we say so.
Method
The method has three parts.
Patient state. We estimate a smooth patient state from the hourly physiological aggregates in MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III. An encoder $E$ maps each hour's partially observed vitals and labs $x_t$ to a vector. The latent state trajectory $z_{1:T}$ then solves
$$\min_{z_{1:T}} \sum_{t \in O} \|z_t - E(x_t)\|^2 + \lambda \sum_{t=2}^{T} \|z_t - z_{t-1}\|^2,$$
where $O$ is the set of hours with observations and $\lambda$ sets how strongly the state resists hour-to-hour change. This is the dynamic word embeddings penalty transplanted to physiology. It gives a state at any note timestamp despite missing hours. $E$ is pretrained on the MIMIC-Extract mortality and intervention tasks, so that $z$ encodes clinically meaningful variation.
Sense scoring. Each candidate sense $s$ gets a vector $g(s)$ by encoding its dictionary definition text with a sentence encoder. We use a MiniLM encoder fine-tuned on synonym pairs from the Unified Medical Language System (UMLS) ontology, as in the efficient biomedical linking paper. A mention $i$ of an abbreviation at time $t_i$ gets a sense distribution $p_i(s) \propto \exp\big(f(\text{text}_i)^\top g(s) + h(z_{t_i})^\top g(s)\big)$. Here $f$ encodes the note snippet and $h$ projects the patient state into the same space. Unseen senses are scored the same way as seen ones, because they enter only through their definitions.
The physiology-gated penalty. This is the new part. For mention pairs $(i, j)$ of the same abbreviation in the same stay, we add
$$\mathcal{L}_{\text{gate}} = \sum_{(i,j)} w_{ij}\,\|p_i - p_j\|^2, \qquad w_{ij} = \exp\!\Big(-\tfrac{|t_i - t_j|}{\eta}\Big)\exp\!\Big(-\tfrac{\|z_{t_i} - z_{t_j}\|^2}{\sigma^2}\Big),$$
where $\eta$ is a time bandwidth in hours and $\sigma$ is a physiology bandwidth. The full objective is cross-entropy on labeled mentions of training abbreviations plus $\mathcal{L}_{\text{gate}}$ over all mentions, labeled or not, including those of held-out abbreviations. A single confident mention, for example one with a disambiguating phrase such as "RF, on HD", then pulls nearby, physiologically consistent bare mentions toward the same sense. Setting $\sigma \to \infty$ recovers ungated "one sense per stay" smoothing. Removing the term recovers a purely supervised model.
Data and Evaluation Sets
We need abbreviation mentions that carry gold senses, timestamps, and aligned vitals. No existing benchmark in our sources provides all three.
Reverse-substitution set. Following MeDAL's reverse-substitution method, we take MIMIC-III ICU nursing and physician notes and replace expanded phrases ("renal failure", "respiratory failure", "pulmonary embolism") with their abbreviations. The original phrase is the gold sense, and the note's chart time aligns it to MIMIC-Extract vitals.
Manually annotated set. Because substitution creates artificial contexts, we also annotate a few thousand naturally occurring abbreviations in the same notes.
Cold-start split. We hold out roughly a third of abbreviations entirely, removing all of their labels from training while keeping their unlabeled mentions. We also define a sense-switch subset: stays in which the same abbreviation carries two different gold senses.
Pretraining. MeDAL is used to pretrain the text encoder $f$.
Off-domain sanity check. If the ICU ground truth from the task-oriented-resources paper and the VUH/UMN sets from Clinical Abbreviation Disambiguation Using Neural Word Embeddings are obtainable, we evaluate the text-only part of our model on them. These sets lack vitals, so they cannot test the main claim.
Baselines
Naive reference. The do-nothing reference assigns each abbreviation its most popular dictionary sense. The task-oriented-resources paper shows dictionary popularity is a strong prior; it is part of their hybrid ranker.
Strongest text-only competitor. This is our own text and definition matcher with no physiology and no penalty: the MeDAL-pretrained encoder $f$ scored against UMLS-tuned definition vectors $g$. It combines the two best ingredients in our sources, MeDAL pretraining and the definition-space reranking of Efficient Biomedical Entity Linking, which reports R@1 of 0.661 on MedMentions st21pv. It is the strongest competitor because it already handles unseen senses through definitions, which is exactly what our penalty must improve on.
For orientation only, the strongest reported abbreviation result in our sources is 82.27% accuracy on the task-oriented-resources paper's ICU ground truth. That number comes from a different dataset without vitals. We found no reported result on Lacuna for any method on MIMIC-III abbreviation disambiguation with aligned physiology, or on a cold-start abbreviation split.
Strongest feature-based competitor. This model is the text matcher plus a feature path: current MIMIC-Extract vitals, imputed by forward filling, are fed directly into the scorer.
Ablations. Three ablations isolate the key ingredient. The first uses the smoothed state $z$ in the scorer but drops $\mathcal{L}_{\text{gate}}$. The second keeps the penalty with $\sigma \to \infty$, which is ungated time smoothing. The third replaces each patient's physiology with that of a randomly chosen other patient whose stay has a similar length.
Fairness. All systems share the same encoders, the same training notes and unlabeled pool, and the same tuning budget of 30 configurations each, selected on a validation set of seen abbreviations only.
What counts as winning. Beating the strongest baseline means higher accuracy on cold-start abbreviations than both the text matcher and the feature-concatenation model, with the gap exceeding the 95% bootstrap interval. We set the target at a few points of absolute accuracy. In addition, on the sense-switch subset, the gated model must be no worse than the text matcher, while the ungated model is expected to fall below it.
Why not just…?
Why not just give a clinical language model the whole patient record? The efficient biomedical linking paper found that adding window or whole-document context to sparse clinical mentions frequently lowered linking accuracy. PubMed-trained representations also systematically choose the wrong ICU senses, scoring 28.06% on ICU abbreviations. A long-context text model is still a reasonable threat on seen abbreviations. We therefore include a variant of the text matcher that reads the preceding 24 hours of notes. We expect its advantage to shrink on held-out abbreviations, where no supervised signal ties the longer text to the right sense. If it matches our model there, the conclusion is that text history is enough, and we will report that.
Why not just add the vital signs as input features? This is the most serious alternative. On abbreviations with abundant labels it may well match our model, because the classifier can learn which physiology goes with which sense. For held-out abbreviations it has no labels from which to learn that link. Our penalty instead transfers confident predictions between mentions without labels. The comparison between the feature-concatenation model and the no-penalty ablation, split by seen versus held-out abbreviations, settles this directly. If concatenation matches the full model on held-out abbreviations, the claim fails.
Why not just assume one sense per patient stay and take a majority vote? This simple heuristic captures much of what our penalty does, and our ungated ablation is its soft version. It will likely deliver most of the average gain. Its failure mode is stays where the sense genuinely changes: "PE" can mean physical exam in an admission note and pulmonary embolism after a desaturation event. The sense-switch subset measures exactly this. If gating gives no benefit there, the physiology part of the proposal is unnecessary and the simpler heuristic should be preferred.
Why not just let a recurrent network learn the dynamics instead of adding an explicit penalty? The temporal knowledge base completion paper reports that an explicit smoothness prior beat recurrent evolution of embeddings at lower cost, and the tgLASSO work found that explicit fused smoothness recovered biomarkers missed by unconstrained models. More to the point, a recurrent network over vitals still needs labels to connect the hidden state to senses. The penalty's role is to supervise unlabeled mentions, which no architecture provides on its own.
Proposed Experiments
The primary experiment trains all systems on the seen-abbreviation labels plus the full unlabeled pool. It reports accuracy and macro-averaged accuracy over senses on four slices: held-out abbreviations, seen-but-rare abbreviations (fewer than 20 labels), seen-frequent abbreviations, and the sense-switch subset. Results are reported separately for the reverse-substitution set and the manually annotated set, so that substitution artifacts cannot drive the conclusions.
Three checks decide the claims. First, the shuffled-physiology control must erase most of the gated model's advantage over ungated smoothing. If it does not, the gate is acting as a generic regularizer rather than reading physiology. Second, accuracy should rise with the density of a stay's mentions. That is, the gain on held-out abbreviations should be larger in stays with more mentions of the same abbreviation, which is the signature of propagation along the timeline rather than of better features. Third, a sweep over $\eta$ and $\sigma$ should show the over-dampening that the dynamic word embeddings paper warns about: very wide bandwidths should hurt the sense-switch subset while helping the average. This tells practitioners when gating is worth its cost.
As a secondary check that the learned states mean something clinically, we test whether $z$ still predicts MIMIC-Extract mortality and interventions near the reported 0.85 to 0.90 mortality AUROC. This confirms that the smoothing in the first equation did not wash out the physiology. A negative outcome on the primary experiment would still be informative: it would show that, for clinical shorthand, physiology adds nothing beyond a well-matched definition encoder.
References
- Exploiting Task-Oriented Resources to Learn Word Embeddings for Clinical Abbreviation Expansion
- Dynamic Word Embeddings for Evolving Semantic Discovery
- Tensor Decompositions for Temporal Knowledge Base Completion
- Temporal Subspace Clustering for Human Motion Segmentation
- Out-of-Sample Extension for Dimensionality Reduction of Noisy Time Series
- Context-Aware Feature and Label Fusion for Facial Action Unit Intensity Estimation With Partially Labeled Data
- Temporally-Constrained Group Sparse Learning for Longitudinal Data Analysis
- Predict multi-type drug–drug interactions in cold start scenario
- Bridging chemical structure and conceptual knowledge enables accurate prediction of compound-protein interaction
- MOTIVE: A Drug-Target Interaction Graph For Inductive Link Prediction
- Named Entity Recognition in Chinese News Comments on the Web
- ERNIE: Enhanced Language Representation with Informative Entities
- Efficient Biomedical Entity Linking: Clinical Text Standardization with Low-Resource Techniques
- MeDAL: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretraining
- MIMIC-Extract: a data extraction, preprocessing, and representation pipeline for MIMIC-III
- Clinical Abbreviation Disambiguation Using Neural Word Embeddings