Reward-Guided Generation Improves the Scientific Utility of Synthetic Biomedical Data

Nicholas J Jackson, Natalia Espinosa-Dice, Chao Yan, Bradley MalinmedRxiv 2026 · DOI

TL;DR

Researchers have developed RLSYN+REG, a reinforcement learning-based generative model that ensures synthetic biomedical data preserves the specific statistical relationships needed for scientific analysis. By rewarding the generator when its output matches the regression patterns of real data, the model significantly improves "scientific utility"—closing the predictive performance gap by up to 97%—without compromising participant privacy.

What Problem Does This Paper Address?

Synthetic data is a "holy grail" for biomedical research: it allows scientists to share datasets without exposing sensitive patient information and to augment small datasets (like those for rare diseases). However, most current generative models (like standard GANs) focus on fidelity—making the data look real on the surface.

The problem is that data can look realistic but fail to preserve the underlying statistical associations (e.g., the relationship between a drug dosage and mortality). If a researcher runs a regression on synthetic data and gets different coefficients than they would on real data, the synthetic dataset has low scientific utility and can lead to misleading conclusions. Existing fixes often require complex causal graphs or simple, rigid rules that don't capture the nuance of medical research.

Methods

The authors build upon RLSYN, a framework that treats synthetic data generation as a Reinforcement Learning (RL) problem. In this setup:

  • The Generator acts as an agent (policy) that learns to produce data rows.
  • The Discriminator acts as a critic, rewarding the generator for "realism."

The key innovation in this paper is the Regression-Based Reward ($R_{reg}$). Before training the generator, the authors fit a regression model ($f$) on the real data. During training, the generator receives an additional reward based on how closely its internal probability of an outcome (like mortality) matches the prediction from the real-data regression model.

Mathematically, the penalty is the squared difference: $R_{reg}(x) = -(q(x) - f(x))^2$. This "steers" the generator to produce data that respects the scientific relationships found in the original dataset. Because they use RL (specifically Proximal Policy Optimization or PPO), the reward function doesn't need to be differentiable, making the framework highly flexible.

Datasets

The model was evaluated on two distinct tabular datasets:

  1. MIMIC-III: A publicly available critical care database. The authors used a cohort of 27,594 ICU admissions to predict in-hospital mortality based on 66 continuous variables (vitals, labs) and several categorical features (race, insurance, etc.).
  2. American Community Survey (ACS): A dataset of 54,452 participants from Tennessee (2018). The task was to predict the receipt of public income assistance based on socioeconomic factors like education, age, and disability status.

Key Insights and Ideas

  • Utility vs. Fidelity Trade-off: There is a slight "tension" between making data look perfectly like the original distribution and making it scientifically useful. RLSYN+REG accepts a very small hit to general fidelity to gain a massive boost in regression accuracy.
  • Decoupling Architecture from Objectives: Because the reward is treated as a "black box" signal in RL, researchers can swap out the regression reward for other goals (like preserving demographic parity or specific odds ratios) without ever changing the underlying neural network architecture.
  • Theoretical Guarantee: The authors provide a mathematical proof showing that if the generator matches the conditional probabilities of the real-world regression model, the synthetic data is guaranteed to recover the same coefficients as the real data.
  • Robustness to Data Scarcity: The model maintains its utility advantage even when training data is reduced, which is critical since synthetic data is most needed in "data-scarce" environments like rare disease research.

Key Results

The improvements in scientific utility were substantial across both datasets:

  • Coefficient Correlation: On MIMIC-III, the correlation between regression coefficients from real vs. synthetic data jumped from 0.054 (base RLSYN) to 0.600 (RLSYN+REG).
  • Predictive Performance: The model reduced the gap between real-data and synthetic-data performance by 81.4% for MIMIC-III (AUC) and 97.6% for ACS (RMSE).
  • Privacy Preservation: Despite being "steered" toward real-data patterns, the model showed no increase in privacy risk. Membership Inference Attack (MIA) scores remained near 0.5 (random chance), meaning an attacker couldn't tell if a specific individual's data was used to train the model.

Limitations and Future Work

  • Tabular Data Only: This study focused on structured, tabular data. Extending this reward-guided approach to more complex modalities like longitudinal electronic health records (EHR) or medical imaging remains an open challenge.
  • Population-Level Focus: The current regression reward optimizes for the whole population. The authors suggest that future work could target subgroup-specific rewards to ensure that synthetic data accurately represents health disparities or minority populations.
  • Bias Amplification: Since the model is rewarded for matching real-data patterns, it will faithfully reproduce any biases present in the original data. The authors note that future rewards could be designed to "de-bias" the synthetic data instead of just replicating the status quo.

Summary updated

Abstract

Abstract Synthetic data generation is a promising approach for biomedical data sharing and dataset augmentation, yet existing methods lack mechanisms to preserve statistical properties necessary for scientific analysis. To address this, we introduce RLSYN+REG, a reinforcement learning-driven generative model, which encourages that regression models trained on synthetic data reproduce the coefficients and predictions of their real-data counterparts. We evaluate RL-S yn +R eg on MIMIC-III and the American Community Survey (ACS) across regression model reproduction, fidelity to real data, and privacy. Synthetic data from RLS yn +R eg substantially improves upon that of RLS yn , raising correlations between real and synthetic regression coefficients from 0.054 to 0.600 on MIMIC-III and from 0.160 to 0.376 on ACS. Predictive performance also improves, reducing the gap between real-data baselines by 81.4% and 97.6% on MIMIC-III and ACS, respectively. These improvements come with negligible cost to fidelity or privacy and are robust to reductions in training data.
BibTeX
@misc{jackson2026reward,
  author = {Nicholas J Jackson and Natalia Espinosa-Dice and Chao Yan and Bradley Malin},
  title = {{Reward-Guided Generation Improves the Scientific Utility of Synthetic Biomedical Data}},
  year = {2026},
  doi = {10.64898/2026.03.11.26348077},
  howpublished = {medRxiv},
}
Figure