Hospitals increasingly use language models to read clinical notes and pull out social facts about patients: whether they have stable housing, a ride to follow-up visits, someone at home to help. Those extracted facts are then fed into a logistic regression. Logistic regression is the standard clinical model that turns risk factors into adjusted odds ratios, meaning "how much this factor multiplies the odds of the outcome, holding everything else fixed." The resulting numbers are what clinicians and health systems act on. For example, one recent study reports that limited transportation doubles 30-day heart-failure readmission odds (OR 2.07). The problem is that the extractors make systematic, predictable mistakes. A behavioral audit of note-based extractors found they fail 44.5% of negation tests ("denies housing problems" read as a housing problem) and 70.9% of attribution tests (a family member's circumstances assigned to the patient). An odds ratio estimated on mislabeled inputs can shrink toward no effect, inflate, or even flip sign. Nobody currently checks.
We propose a risk model built to make those odds ratios trustworthy. Social facts extracted from notes become explicit, named inputs to a multivariable logistic readout, and each one carries a known error profile. That error profile is measured against human annotation, broken down by error type, and corrected for inside the model's likelihood. High-dimensional physiological data (waveforms, labs, vitals) enter the same readout, but only as a few compact risk scores. The deep networks behind those scores are trained on short-horizon, biologically driven targets and then frozen. This keeps them from quietly absorbing social variance. The deliverable is a readmission and revisit model whose social coefficients match those obtained from gold-standard labels, and whose physiological inputs cannot drown out the social ones.
Core Claim
The obvious combination of these four directions would be a multimodal concept bottleneck: a network forced to predict human-readable intermediate concepts before the final output, here with social concepts in the bottleneck and a logistic head on top. The research file contains nothing that builds this for social determinants. Its closest neighbors are Multimodal Concept Bottleneck Models, which uses general vision-language concepts, and Integrating Clinical Knowledge into Concept Bottleneck Models, which uses single-modality imaging concepts.
Building that model is not enough, though, because a skeptic can reproduce most of it with a two-stage pipeline. Mining Social Determinants of Health for Heart Failure Patient 30-Day Readmission via Large Language Model did exactly that: zero-shot Llama-3.1-8B extraction, then an offline logistic regression. Our claim is narrower and testable. The two-stage pipeline produces biased social odds ratios because extraction errors are structured, not random. Two specific ingredients remove the bias without giving up predictive performance: a misclassification-aware likelihood, and physiological scores frozen on acute targets. If the bias turns out to be negligible, the two-stage pipeline is vindicated. That would also be worth knowing.
Research Program
The model has three kinds of input. The first is note-derived social concepts. We use the seven categories of MIMIC-SBDH: A Dataset for Social and Behavioral Determinants of Health: community, economics, education, environment, alcohol, tobacco, and drugs. We add transportation, which the Shao study found informative. Each concept is scored by an extractor, and the extractor must output a probability rather than a hard label.
The second is area-level context. These are the County Health Rankings, Social Vulnerability Index, and AHRQ features linked through patient zip code, as in Evaluating the Impact of Social Determinants on Health Prediction. That paper found such features "accuracy-neutral" but able to cut in-hospital mortality calibration error from 0.11 to 0.03. We therefore include them as calibration anchors, not as signal carriers.
The third is two or three physiological risk scores. Each comes from an encoder over vitals, labs, and (where available) waveforms. The encoder is trained on an acute target such as in-hospital deterioration and then frozen. The rationale comes from Hogan et al., who found that early readmissions are driven by biology while later readmissions are driven by transportation and adherence. An encoder trained on biology-dominated targets is less likely to encode social proxies than one trained directly on long-horizon readmission. Compressing each modality into a scalar also addresses the dominance problem documented for image-plus-table fusion, where a large modality's gradients drown out a 20-feature table, as seen in TabMixer and TFormer. This design constrains, rather than borrows from, those architectures: we give up their rich cross-modal mixing in exchange for a readout whose coefficients mean something.
The central mechanism is how the social concepts enter the likelihood. Let $c$ be the true binary social state, $\tilde{c}$ the extractor's output, $x$ the remaining covariates (physiological scores, area-level features, demographics), and $Y$ the outcome. The model is
$$P(Y=1 \mid \tilde{c}, x) = \sum_{c \in \{0,1\}} \sigma(\beta_0 + \beta_c c + \beta_x^\top x)\, P(c \mid \tilde{c}, e),$$
where $\sigma$ is the logistic function and $\beta_c$ is the social log-odds ratio we want to recover. The term $P(c \mid \tilde{c}, e)$ is the extractor's calibrated misclassification profile, estimated from human-annotated notes. It is conditioned on $e$, the linguistic context flagged in the sentence: negation cue present, a family-member mention nearby, or neither. Conditioning on $e$ is what separates this from generic measurement-error correction. The Gaur audit shows that errors concentrate in exactly these contexts, so a single overall confusion matrix would under-correct. Standard errors for $\beta_c$ must account for the fact that the confusion profile is itself estimated. We handle this by bootstrapping over the annotated calibration set.
Baselines
The naive reference point is a multivariable logistic regression on structured EHR covariates and physiological summaries only, with no social information at all. Every other model must beat it on long-horizon outcomes to justify its complexity.
The first social baseline follows the Yang et al. design: the same logistic regression plus zip-code-linked area features, run on the same splits. Since that paper reports no significant AUROC or AUPRC change, we expect parity in discrimination. The comparison that matters is calibration error, where the reported in-hospital mortality figure is 0.11 without these features and 0.03 with them.
The strongest competitor on our specific question, trustworthy social odds ratios, is the Shao two-stage pipeline: zero-shot Llama-3.1-8B extraction with hard labels, followed by offline multivariable logistic regression. Its reported numbers are extraction accuracy (housing 87.14%, tobacco 79.19%) and odds ratios (transportation 2.07, social support 0.58). Neither is a benchmark score we can beat head-on. We will re-run it with the same extractor, prompts, cohort, and covariates as our model, so the only difference is the correction term. Our model beats it if its social odds ratios are closer to gold-label odds ratios and its confidence intervals cover the gold value at close to the nominal rate, without losing more than a trivial amount of AUROC.
The strongest predictive competitor is the MC-BEC pipeline from Multimodal Clinical Benchmark for Emergency Care. It extracts pretrained embeddings of codes, notes, radiology, and waveforms and feeds them to LightGBM. Its reported revisit AUPRC is 0.11 to 0.24, depending on horizon. We will run it with and without our extracted social concepts appended, using the same hyperparameter search budget as our model.
Two ablations isolate the key ingredients. The first removes the correction term, which recovers a jointly trained two-stage model. The second replaces the frozen acute-target encoders with encoders fine-tuned end to end on readmission. If the second ablation shrinks the social coefficients toward zero while keeping AUROC unchanged, the deep scores are absorbing social variance.
Why not just…?
Why not just use the two-stage LLM-plus-regression pipeline with a better extractor? This is the real threat. If a stronger extractor pushed negation and attribution failures close to zero, the correction would change nothing. Experiment one settles this directly: we measure odds-ratio bias for the two-stage pipeline against gold-label odds ratios. If that bias falls inside the bootstrap noise of the gold estimate, we will report that the two-stage approach is adequate and that extractor quality, not modeling, is the bottleneck. The Gaur failure rates are measured on Bio-ClinicalBERT, not Llama, so we will re-audit Llama-3.1-8B with the same behavioral test types rather than assume those rates carry over.
Why not just feed everything to LightGBM, as MC-BEC does? It may predict revisits as well as or better than our model. We take that seriously and test it. But it yields no adjusted odds ratios. Its "social importance" cannot be separated from correlated physiology, and MC-BEC itself reports modality competition: adding labs and radiology lowered decompensation AUPRC. Our model is judged against it on revisit AUPRC. We accept a small loss there only if we gain coefficient validity and subgroup calibration.
Why not just run SHAP on a black-box fusion model? Post-hoc attributions describe whatever the model used, including shortcuts, as Pang et al. documented for concept models under distribution shift. More basically, attributions computed on mislabeled social inputs inherit the mislabeling. We will compute SHAP values for the black-box baseline's social features and compare their rank and sign against the gold-label odds ratios. This shows directly whether post-hoc explanations recover the true direction of effects.
Why not just use area-level social data, which needs no extraction? Yang et al. show it improves calibration but not discrimination, and they flag the ecological fallacy of zip-code averages. It cannot represent an individual patient's missing ride. We keep it in the model as a covariate, and the first social baseline measures what it contributes on its own.
Proposed Experiments
The decisive experiment is odds-ratio recovery on MIMIC-SBDH. Its 7,025 MIMIC-III discharge summaries carry human status labels, which we link to 30-day readmission for the index admissions. First, we fit the multivariable logistic regression with gold labels plus clinical covariates to obtain reference odds ratios. Next, we hold out a calibration subset of a few hundred notes to estimate the context-conditioned misclassification profile. On the remaining notes, we compare three estimates against the reference: gold-label, naive two-stage, and error-corrected. Outcome measures are absolute log-odds-ratio bias, 95% interval coverage, and sign agreement for each category. We then repeat this after deliberately injecting negation and attribution errors at the rates Gaur reports, to test whether the correction holds under heavier corruption.
The second experiment tests transport to the Shao heart-failure cohort (2,065 patients, 3,604 notes). There is no gold standard there, so we report how far the corrected odds ratios move from Shao's published values. We also annotate a 200-note audit sample to check whether the MIMIC-SBDH-derived error profile transfers to that cohort. If it does not, the method requires site-level annotation, and we will say so.
The third experiment adds the social component to MC-BEC for 3- to 14-day revisits. We extract social concepts from its ED notes, add the frozen waveform-and-vitals scores, and compare revisit AUPRC against the reported 0.11 to 0.24. We also report calibration error and false-positive-rate gaps across age and area-income subgroups. A caveat: ED notes may mention social circumstances far less often than discharge summaries do, so this experiment could show no gain simply because the signal is absent. We will report how often each concept is actually documented, so a null result can be interpreted.
The last experiment is the frozen-encoder ablation described in the Baselines section, run on both cohorts. It will show whether end-to-end physiological encoders absorb social variance, in the sense raised by Chen et al. for psychiatric readmission.
References
- Multimodal Concept Bottleneck Models
- Integrating Clinical Knowledge into Concept Bottleneck Models
- Mining Social Determinants of Health for Heart Failure Patient 30-Day Readmission via Large Language Model
- MIMIC-SBDH: A Dataset for Social and Behavioral Determinants of Health
- Evaluating the Impact of Social Determinants on Health Prediction
- Assessing Predictors of Early and Late Hospital Readmission After Kidney Transplantation
- TabMixer: Noninvasive Estimation of the Mean Pulmonary Artery Pressure via Imaging and Tabular Data Mixing
- TFormer: A throughout fusion transformer for multi-modal skin lesion diagnosis
- Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine
- Can AI Help Reduce Disparities in General Medical and Mental Health Care?