Privacy-aware Synthesizing for Crowdsourced Data

Mengdi Huai, Di Wang, Chenglin Miao, Jinhui Xu, Aidong Zhang

Abstract

Crowdsourcing has emerged as a vital mechanism for large-scale data acquisition, yet the public release of such data often compromises individual privacy. While Differential Privacy (DP) offers a robust theoretical framework for protection, its application to crowdsourced environments typically incurs a heavy "utility tax" due to additive noise, especially in the presence of conflicting reports and outliers. This paper presents PrisCrowd, a novel framework that bypasses traditional noise injection. PrisCrowd employs a two-phase approach: first, it constructs a weighted Kernel Density Estimation (KDE) to represent the underlying truth while accounting for user reliability; second, it generates synthetic data through a sampling process governed by a rigorous privacy test. Experimental results on real-world datasets demonstrate that PrisCrowd significantly outperforms baseline DP methods in preserving data utility while maintaining strong privacy guarantees.

Introduction

The proliferation of crowdsourcing platforms allows for the collection of diverse datasets, ranging from medical side-effect reports to urban planning feedback. However, the utility of this data is often at odds with the privacy of its contributors. Standard Differential Privacy (DP) mechanisms, such as the Laplace or Gaussian mechanisms, add random noise proportional to the sensitivity of the query. In crowdsourcing, where data is often voluminous and contains many conflicting reports due to errors or intentional deception, the required noise often renders the released data statistically useless.

To address this, we propose PrisCrowd (Privacy-aware Synthesizing for Crowdsourced Data). Unlike conventional methods that perturb existing values, PrisCrowd focuses on synthesizing new data points that reflect the aggregate distribution without exposing individual records.

The PrisCrowd Framework

The PrisCrowd architecture is divided into two distinct phases: weighted density estimation and privacy-preserving synthetic release.

The two-phase PrisCrowd architecture: Phase 1 focuses on density estimation from raw claims, while Phase 2 handles synthetic data generation and privacy testing.

Phase 1: Weighted Kernel Density Estimation

Crowdsourced data is inherently noisy. PrisCrowd mitigates this by assigning weights to users based on their reliability:

  1. Global Weight: A macro-level assessment of a user's reliability across all objects they have reported on.
  2. Local Weight: A micro-level assessment for a specific object, utilizing a robust loss function (Hampel loss) to identify and down-weight outliers.
  3. Combined Weight: By merging these weights, the system constructs a weighted KDE for each object. This KDE serves as a probabilistic representation of the "truth" that is more resilient to individual errors than a simple average.

Phase 2: Privacy-Preserving Synthetic Release

Instead of releasing the KDE parameters or the raw data, PrisCrowd generates synthetic claims. This process ensures privacy through a sampling-and-testing loop:

  • Candidate Generation: Synthetic points are sampled using original data records as "seeds."
  • Privacy Test: Each candidate point undergoes a randomized test to ensure it does not uniquely identify its seed. Specifically, a point is only released if it could have been plausibly generated by at least $k$ different potential seed records. This "hiding in the crowd" approach provides formal privacy guarantees without the need for explicit noise addition.

Experimental Evaluation

The framework was evaluated using three real-world datasets: a Population dataset (2,344 users), a Stock dataset (55 users, high outlier count), and an Indoor Floorplan dataset (247 users). Utility was measured using Integrated Squared Error (ISE), which quantifies the discrepancy between the synthetic distribution and the true underlying distribution.

Utility and Robustness

A key strength of PrisCrowd is its ability to maintain high utility even in adversarial conditions. As illustrated in the density plots below, the "Combined" weighting scheme (red line) tracks the true density (blue line) far more accurately than uniform weighting (green line), which is easily skewed by outliers (magenta crosses).

Comparison of density estimation methods across various scenarios. PrisCrowd (red) consistently aligns with the true density (blue), effectively filtering out the noise and outliers that degrade other methods.

On the Stock dataset, which is particularly prone to outliers, PrisCrowd achieved an ISE of 1.699. In contrast, a standard uniform weighting approach resulted in an ISE of 17.628, representing a nearly tenfold improvement in data utility.

Computational Efficiency

While the privacy test introduces an iterative loop, the computational overhead remains manageable. Experiments show that the running time scales linearly with the number of synthetic claims generated, making the framework suitable for large-scale crowdsourcing applications.

Conclusion

PrisCrowd demonstrates that privacy and utility in crowdsourced data release are not mutually exclusive. By shifting from additive noise to a synthesis-and-test paradigm, the framework preserves the statistical integrity of the data while protecting individual contributors. Future work will explore extending this KDE-based approach to higher-dimensional data spaces and investigating decentralized privacy models to further reduce trust requirements on the data collector.

Summary updated

Abstract

Although releasing crowdsourced data brings many benefits to the data analyzers to conduct statistical analysis, it may violate crowd users' data privacy. A potential way to address this problem is to employ traditional differential privacy (DP) mechanisms and perturb the data with some noise before releasing them. However, considering that there usually exist conflicts among the crowdsourced data and these data are usually large in volume, directly using these mechanisms can not guarantee good utility in the setting of releasing crowdsourced data. To address this challenge, in this paper, we propose a novel privacy-aware synthesizing method (i.e., PrisCrowd) for crowdsourced data, based on which the data collector can release users' data with strong privacy protection for their private information, while at the same time, the data analyzer can achieve good utility from the released data. Both theoretical analysis and extensive experiments on real-world datasets demonstrate the desired performance of the proposed method.
BibTeX
@inproceedings{huai2019privacy,
  author = {Mengdi Huai and Di Wang and Chenglin Miao and Jinhui Xu and Aidong Zhang},
  title = {{Privacy-aware Synthesizing for Crowdsourced Data}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/353},
}
Figure