Privacy-Aware Data Integration for Enhanced Quantile Inference under Heterogeneity

Leheng Cai, Qirui Hu, Shuyuan Wu

Introduction

Quantile estimation—such as calculating the median or the 95th percentile—is a cornerstone of risk management, finance, and healthcare. While statistical estimates can be significantly improved by pooling data from multiple organizations, two major hurdles often stand in the way: privacy and heterogeneity.

First, legal and ethical barriers frequently prevent institutions from sharing raw data. Instead, they must rely on privacy-preserving mechanisms like Local Differential Privacy (LDP), which adds noise to the data at its source. Second, different sites are rarely identical; a hospital in an urban center will likely have a different patient demographic than a rural clinic. Existing methods often struggle with the "non-smooth" nature of quantile loss functions under these privacy constraints, or they fail to provide valid statistical inference when the differences between sites (parameter shifts) are subtle but significant.

This paper introduces a robust, systematic framework for integrating data from multiple sites to improve quantile estimation while strictly adhering to LDP. By leveraging adaptive weighting and a novel parallel stochastic gradient descent (PSGD) algorithm, the authors enable highly accurate statistical inference even across heterogeneous data sources.

Methodology: A Three-Pillar Framework

The authors propose a systematic approach centered on three key methodological innovations designed to handle both privacy constraints and site-to-site differences.

1. Adaptive Weighted Estimator

Instead of treating all data sources equally, the proposed model assigns adaptive weights to different sites. This is achieved by minimizing a loss function that balances asymptotic variance against an $\ell_2$ penalty for parameter shifts (bias). Consequently, the model automatically down-weights "noisy" or "dissimilar" sites, ensuring that the integration of auxiliary data only enhances, rather than degrades, the target estimate.

2. Parallel Stochastic Gradient Descent (PSGD)

Quantile loss functions are notoriously "pointy" or non-smooth, which typically complicates standard variance estimation. To estimate the variance of the privatized data without accessing the raw distribution, the authors partition each site's data into multiple independent "chains." By running Stochastic Gradient Descent (SGD) on these chains in parallel, they can estimate the variance of the final estimator directly from the variation between chains. This bypasses the need to estimate the underlying probability density function, making the method highly practical for real-world LDP applications.

3. Conservative Weighted Estimator

The authors identify a statistical "danger zone" where the bias between sites is roughly the same size as the standard error ($\mathcal{O}(N^{-1/2})$). In this regime, standard adaptive weights can lead to overconfidence and invalid inference. To counter this, they introduce a conservative weighted estimator that utilizes an upper bound on bias, ensuring that confidence intervals remain valid even under moderate heterogeneity.

Experimental Validation

The framework was rigorously evaluated using both synthetic environments and real-world datasets.

  • Synthetic Data: The authors generated data using Normal and Cauchy distributions. The Cauchy distribution is particularly relevant as it is "heavy-tailed," representing a classic scenario where quantile estimation is far more informative than standard mean estimation.
  • Real-World Data: The framework was applied to the Government Salary Dataset (derived from the 2018 American Community Survey). By treating different US "economic regions" as separate sites, the authors estimated income quantiles, accurately simulating a scenario where sensitive income data is protected by strict privacy laws.

Key Results

  • Efficiency Gains: In simulations, the Adaptive (ADP) methods consistently produced lower Mean Squared Error (MSE) and narrower confidence intervals compared to baselines that only used the target site's data.
  • Robust Coverage: While simpler integration methods (like Inverse Variance Weighting) saw their coverage drop below the nominal 95% when sites were heterogeneous, the conservative estimator (ADP-cons) maintained valid 95% coverage across all tested bias levels.
  • Real-World Utility: On the salary dataset, the integrated approach provided significantly tighter confidence intervals for income quantiles than the single-site baseline. This demonstrates that auxiliary data can successfully "sharpen" our understanding of sensitive populations without compromising individual privacy.

Key Insights

  • The "Blessing" of Mild Heterogeneity: The paper provides theoretical proof that if at least one source site has a bias smaller than the statistical noise, the integrated estimator will always be more efficient than relying on the target site alone.
  • Privacy-Accuracy Trade-off: The mathematical framework explicitly links the "truthful response rate" (the inverse of the noise added for LDP) to the final variance. This provides researchers and practitioners with a clear, quantifiable dial to tune the balance between strict privacy and high-precision results.

Limitations and Future Work

While the proposed framework represents a significant step forward for privacy-aware data integration, the authors acknowledge a few areas for future exploration:

  • Asymptotic Requirements: The current theoretical guarantees rely on the number of PSGD chains growing with the sample size. Future work could explore "fixed-chain" settings, which would be highly beneficial for smaller datasets.
  • Number of Sites: The analysis currently assumes a fixed number of sites. Extending the theory to accommodate a "massive" number of sites (a diverging number of sources) remains an open challenge.
  • Complex Environments: Although validated on census salary data, broader domain-specific evaluations—such as in complex, multi-center healthcare networks or autonomous vehicle fleets—are needed to fully vet the framework's robustness in the wild.

Summary updated

Abstract

Quantile estimation and inference play essential roles in diverse scientific and industrial applications, and their accuracy can often be enhanced by integrating auxiliary data from multiple sites. However, developing efficient aggregation methods for quantile inference under potential privacy constraints, particularly with heterogeneous datasets, remains challenging. To address these issues, we propose a systematic framework for quantile estimation and inference under potential local differential privacy (LDP). The key idea is to construct weighted estimators by adaptively aggregating quantile estimates from target and source sites. The adaptive weights are determined by minimizing the asymptotic variance, incorporating an additional $\ell_2$ penalty to account for parameter shift. A parallel stochastic gradient descent algorithm under LDP constraints is developed for weight estimation and valid inference. Additionally, we introduce a conservative weighted estimator to ensure robust inference across diverse heterogeneous scenarios. Rigorous theoretical analysis establishes the consistency, normality, and effectiveness of the proposed methods. Extensive numerical studies and real data application corroborate our theoretical findings.
BibTeX
@inproceedings{cai2026privacy,
  author = {Leheng Cai and Qirui Hu and Shuyuan Wu},
  title = {{Privacy-Aware Data Integration for Enhanced Quantile Inference under Heterogeneity}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=f3q85c5wkg},
}
Figure