Releasing Private Synthetic Survey Data That Stays Correct Under Self-Report Bias

People misreport sensitive behaviour in predictable directions. Heavy drinkers report drinking less, patients report taking their medication more faithfully, and adolescents under-report self-harm. Those surveys are also the data that privacy law keeps locked inside institutions. A common way to share such data is to release a differentially private synthetic copy: an artificial table whose statistics come from the real one, with calibrated random noise added so no individual can be identified. Recent work shows that the added noise alone breaks ordinary statistics. Regression slopes shrink toward zero, and in the high-privacy regime nominal confidence intervals can cover the true value 0% of the time. A known-noise correction can repair this. That repair assumes the underlying values were recorded truthfully. For self-reported behaviour they were not. The released table therefore carries two errors stacked on top of each other: the human's directional misreport and the curator's random privacy noise. Fixing only the second yields an intervals that is statistically valid around the wrong answer.

This note proposes a release protocol and estimator for stacked error. The data curator holds the raw survey together with objective proxies, such as biomarkers or administrative records, that track the true behaviour but are not driven by the urge to look good. On site, the curator fits a model of how respondents distort their answers. The curator then releases three things: the privacy-protected synthetic table of reported values, a privacy-protected summary of the distortion model, and its uncertainty. An outside analyst who receives only these public objects can compute effect estimates and confidence intervals that are valid for the true behaviour. In plain terms, the release makes an outside replication of a sensitive behavioural study both possible and statistically honest.

Core Claim

Under a stated distortion model, an analyst who combines a known-noise privacy correction with a separately released, privacy-protected estimate of self-report bias can recover near-nominal confidence-interval coverage for regression effects of a misreported exposure. A privacy-only correction cannot, and its failure grows with the size of the bias, not with the privacy budget. The main thing to test is whether the joint release is better at a fixed total privacy budget than the obvious alternative, in which the curator corrects the records first and then releases a synthetic copy of the corrected values.

Write $X$ for a person's true exposure, for example weekly alcohol intake. $R$ is what they report, $\tilde R$ is the value in the private synthetic release, and $Y$ is the outcome whose dependence on $X$ we want, summarized by a slope $\beta$. The error structure is

$$R = X + \alpha\, h(X, C) + \eta, \qquad \tilde R = R + U, \quad U \sim \mathcal N(0, \sigma_U^2).$$

Here $C$ are covariates, $h$ is a known shape for how misreporting depends on the true level and on context, $\alpha$ is the unknown bias strength, $\eta$ is ordinary reporting noise, and $\sigma_U$ is fixed by the privacy mechanism and therefore public. The distortion term is directional and depends on $X$, so adding more respondents does not average it away. The privacy term is exogenous and fully known. The proposal's estimator uses the known $\sigma_U$ together with a released estimate $\hat\alpha$ and its variance. It propagates both into the analyst's interval.

How This Relates to Prior Work

Two papers supply the methods and each fixes one of the two layers. Analysis of Differentially Private Synthetic Data: A Measurement Error Approach treats private synthetic data as a classical errors-in-variables problem with exactly known additive noise. It derives a bias-corrected maximum-likelihood estimator for linear regression that restores 90% coverage where naive intervals collapse. It explicitly assumes the pre-noise values are correct, and it leaves logistic and other generalized linear models open. We keep its known-noise correction as our second stage, drop the truthful-input assumption, and extend the analysis to logistic outcomes.

Proxy-Guided Measurement Calibration separates a true outcome from systematic reporting bias. It uses proxies that are conditionally independent of the bias mechanism, a two-stage variational autoencoder, and a matching estimator for the bias parameter $\alpha$. It is evaluated on semi-synthetic OHIE and JOBS data and on the real SHELDUS disaster-loss data. It is a retrospective estimator run by someone with full data access, and it is not a release mechanism. We use it on the curator's side and ask how to publish its output privately so that others can use it. Its additive and monotone bias assumptions constrain our design as well, and we test what happens when they fail.

Several papers motivate the problem. Global burden of lip and oral cavity cancer attributable to high alcohol consumption from 1990 to 2021 describes how under-reporting by heavy consumers shifts dose–response curves and understates risk. Pathways of influence on health-promoting lifestyles in older adults with coronary heart disease documents over-reporting of healthy behaviour. The clinical silo cases, Deep Ensemble Learning for the Automatic Detection of Pneumoconiosis in Coal Worker's Chest X-ray Radiography and A community effort for automatic detection of postictal generalized EEG suppression in epilepsy, show that without a shareable release, reported results such as 91.50% accuracy cannot be checked by anyone else. MIMIC-III, a freely accessible critical care database is the main counterexample of successful open release. Its de-identification still passes self-reported fields through unchanged, and it comes from a single center.

The file also contains work on LLM-generated synthetic respondents: Mitigating Social Desirability Bias in Random Silicon Sampling, Large Language Models Show Human-like Social Desirability Biases in Survey Responses, and Exploring Social Desirability Response Bias in Large Language Models. These papers study bias introduced by the generator. We study bias already present in human data before any generator touches it. This proposal therefore does not use LLM generation, and the "Why not just" section explains why.

Research Program

Curator-side estimation. The curator fits the proxy-guided calibration model to the raw data to obtain $\hat\alpha$, the fitted shape $h$, and a variance for $\hat\alpha$. For the variance we use a bootstrap over the matching estimator, because the source paper reports identification results but no interval for $\alpha$.

Private release. The total privacy budget $\epsilon$ is split. The fraction $\epsilon_1$ goes to the synthetic table of reported values, generated with the analytic Gaussian mechanism so that $\sigma_U$ is exactly known. The fraction $\epsilon_2$ goes to a privatized release of the low-dimensional bias summary ($\hat\alpha$, the parameters of $h$, and their variance). The bias summary has only a handful of numbers, so we expect it to need a small $\epsilon_2$. Whether that holds is an empirical question, and the budget split is a tunable choice we will report.

Analyst-side estimator. For linear outcomes we extend the closed-form estimator of the differential-privacy measurement-error paper. The extension integrates over the conditional distribution of $X$ given $\tilde R$ implied by both error layers. We then add a variance term for the uncertainty in $\hat\alpha$, estimated by the delta method and checked by parametric bootstrap. For logistic outcomes, a closed form is unlikely. We will fit the likelihood by Monte Carlo integration over $X$ and compare it against simulation-extrapolation as a cheaper fallback.

Robustness to a wrong distortion model. Real misreporting is rarely additive. We will test multiplicative under-reporting that grows with true intake, which is the pattern the alcohol-burden paper describes, and threshold-style denial, in which some users report zero. The goal is to map where the method degrades rather than to claim it always works.

Baselines

The naive reference point is the analyst running an ordinary regression of $Y$ on $\tilde R$ with standard intervals. It shows the combined size of both failures. Prior work already predicts coverage near 0% at low $\epsilon$ from privacy noise alone.

The first strong baseline is the privacy-only correction from Analysis of Differentially Private Synthetic Data: A Measurement Error Approach, applied exactly as published to the same synthetic table. It is the best existing method for valid inference from private synthetic data. The reported number is a consistent nominal 90% coverage rate on its own simulations, where naive intervals often show 0% coverage at low $\epsilon$. That result assumes truthful inputs. We run it with the full budget $\epsilon$ on the table, so it gets strictly more privacy budget for the table than our method does. Beating it means our intervals stay within a few points of 90% coverage while its coverage falls as $\alpha$ grows, at matched total $\epsilon$.

The second strong baseline, and the most serious competitor, is calibrate-then-synthesize. The curator applies Proxy-Guided Measurement Calibration to impute a corrected exposure for each record. It then releases a private synthetic copy of the imputed values, and the analyst applies the privacy-only correction. To make it fair, we give it the same total $\epsilon$, the same proxy model and tuning budget, and a strengthened form: multiple imputation, with $m$ corrected tables drawn from the bias model's posterior and each privatized at $\epsilon/m$, combined by standard multiple-imputation rules. Beating it means equal or better coverage with narrower intervals at matched $\epsilon$. Our method would also keep the reported variable available for analyses of misreporting itself. The file contains no reported numbers for this baseline because it has not been evaluated anywhere on Lacuna. The proxy-calibration paper reports no numerical bias-recovery error on OHIE or JOBS, only that it "significantly outperforms baselines" at moderate to high bias.

The key ablation is our estimator with $\hat\alpha$ plugged in as a fixed number, without propagating its variance. It isolates the claim that releasing the uncertainty of the bias model is what restores coverage. If the plug-in version matches the full version, the variance-propagation machinery is unnecessary and we will say so.

Why not just…?

Why not just correct the records before synthesizing? This is the real threat, and it may win for point estimates. Its weaknesses are specific. A single imputed table treats estimated corrections as if they were observed, so intervals should be too narrow. Multiple imputation fixes that but splits the privacy budget $m$ ways over a full table, which multiplies noise. Our release spends the extra budget only on a few summary numbers. The matched-budget comparison in the experiments settles this directly. If calibrate-then-synthesize with multiple imputation matches our coverage and width, our contribution shrinks to keeping the reported variable in the release, and we will report that plainly.

Why not just release the objective proxy and skip self-report? If a biomarker were the construct of interest, this would be the right answer. Predicting suicidality with small sets of interpretable reward behavior and survey variables shows objective tasks can reach 0.80–0.95 AUC. It also shows they reach peak accuracy only together with survey variables, and its labels are themselves self-reported. The proxy in our setting is a correlate of the exposure, not the exposure. We will include a proxy-only regression as a baseline and report how far its slope sits from the true $\beta$ on semi-synthetic data where $\beta$ is known.

Why not just ask respondents to be honest, or generate the data with an LLM? Both ideas fail on the evidence in the file. Honesty instructions increased measured social desirability from 4.60 to 5.49 in Exploring Social Desirability Response Bias in Large Language Models. Priming and preamble prompts worsened divergence from human answers in Mitigating Social Desirability Bias in Random Silicon Sampling. In humans, prompts about awareness produce endorsement without matching practice, as in Knowledge, attitude and practice towards glaucoma among ophthalmic inpatients. LLM respondents also add their own bias: Large Language Models Show Human-like Social Desirability Biases in Survey Responses reports Conscientiousness inflated by about 1.20 standard deviations. Neither approach addresses bias already present in existing human data.

Proposed Experiments

We first run fully simulated experiments that mirror the design of the privacy-correction paper, with $N$ from 500 to 100,000 and $\epsilon$ from 0.1 to 5. We add a bias strength $\alpha$ swept from zero to large, and both linear and logistic outcomes. Knowing the true $\beta$, we measure absolute bias of $\hat\beta$, coverage of nominal 90% intervals, and interval width for every method in the Baselines section.

We then run semi-synthetic experiments on the OHIE Medicaid lottery data and the JOBS training data used by Proxy-Guided Measurement Calibration. We inject misreporting of known $\alpha$ into one self-report exposure, use the provided biomedical or earnings proxies, and apply the privacy mechanism. This keeps a real covariate structure while preserving ground truth. SHELDUS has no ground truth, so we use it only to check that the private release reproduces the paper's reported ordering of reporting bias across hazards (floods largest, then tornadoes, wildfires, hurricanes) from public objects alone.

Three results decide whether the idea is worth pursuing. First, at large $\epsilon$ where privacy noise is small, the privacy-only correction's coverage must fall as $\alpha$ rises. If it does not, stacked error is not a practical problem. Second, our estimator must hold near 90% coverage across $\alpha$ and $\epsilon$ under a correctly specified distortion model. Its advantage over calibrate-then-synthesize with multiple imputation at matched total $\epsilon$ must be measurable in coverage or width. Third, under multiplicative and denial-style misreporting we will report how much coverage drops. That defines the scope within which a curator could responsibly use the release.