This paper, presented by researchers from Queen’s University, explores the fundamental tension between data utility and data privacy through the lens of information theory. While most privacy frameworks rely on Mutual Information (MI) to measure how much information is leaked, this paper argues for the use of Maximal Correlation as a more robust alternative for certain privacy-constrained problems.
TL;DR
The authors propose using maximal correlation instead of mutual information to define privacy constraints in data-sharing systems. They derive new theoretical bounds relating these two measures and provide an explicit privacy-preserving scheme using $\alpha$-stable noise that achieves an optimal tradeoff between privacy and estimation error (MMSE).
What Problem Does This Paper Address?
In a distributed data system, we often want to share a version of our data ($Z$) that is useful for a specific task (correlated with a utility variable $Y$) but protects sensitive information (almost independent of a private variable $X$).
The "Rate-Privacy Function" typically uses Mutual Information (MI) to measure both utility and privacy. However, MI has known deficiencies; for instance, it doesn't always capture the "worst-case" functional dependence between variables. If a malicious actor can apply a specific function to the shared data to perfectly recover a private attribute, MI might not fully reflect that risk.
The authors address this by substituting MI with Maximal Correlation in the privacy constraint. Maximal correlation is a more stringent measure of dependence because it looks for the "best" possible correlation between any two functions of the variables.
Methods
The paper is purely theoretical, utilizing tools from probability theory and information theory. The authors' approach follows three main steps:
- Characterization: They define maximal correlation ($\rho_m$) and relate it to the singular values of conditional expectation operators.
- Comparison: They derive new inequalities that bound maximal correlation using Mutual Information for specific distributions (Gaussian, discrete, and binary).
- Privacy Modeling: They introduce a "privacy filter" model where the shared variable $Z$ is a noisy version of the private variable $X$ (specifically $Z = X + \lambda N$). They use $\alpha$-stable distributions (a class of distributions including Gaussian and Cauchy) to model this noise, allowing them to analytically calculate the privacy-utility tradeoff.
Datasets
Not discussed in the paper. This is a theoretical work focused on proving mathematical bounds and properties rather than empirical testing on specific datasets.
Key Insights and Ideas
- Maximal Correlation vs. MI: The paper proves that for jointly Gaussian variables, the square of the maximal correlation is bounded by a constant times the Mutual Information ($\rho_m^2 \leq (2 \ln 2)I(X;Y)$). This provides a bridge between the two metrics.
- The Power of Tensorization: Unlike Mutual Information, maximal correlation has the "tensorization" property. This means that for independent and identically distributed (i.i.d.) data, the maximal correlation of the whole sequence is the same as the maximal correlation of a single sample. This makes it a very stable metric for large-scale data systems.
- $\alpha$-Stable Noise as a Privacy Tool: The authors show that if you use noise from an $\alpha$-stable distribution, you can precisely tune the "privacy knob" ($\lambda$). They provide a closed-form expression for the resulting privacy: $\rho_m = 1 / \sqrt{1 + \lambda^\alpha}$.
- The "Price" of Privacy: The authors quantify utility using Minimum Mean Squared Error (MMSE). They prove that as you tighten the privacy constraint (decreasing the allowed maximal correlation $\epsilon$), the minimum possible estimation error for the utility variable $Y$ necessarily increases.
Key Results
- Theorem 2: Establishes the upper bound for maximal correlation in terms of MI for Gaussian variables.
- Theorem 5: Provides an exact formula for maximal correlation when using $\alpha$-stable noise, showing it is a non-increasing function of the noise scale $\lambda$.
- Theorem 6 (The Main Result): Derives a lower bound for the MMSE of estimating $Y$ from the shared data $Z$, given a privacy constraint $\epsilon$. Specifically: $$mmse_\epsilon(Y) \geq (1 - \varrho_\epsilon^2(X; Y))var(Y)$$ This formula gives engineers a concrete way to calculate the best possible utility they can expect for a given level of privacy.
Limitations and Future Work
- Infinite Support: The $\alpha$-stable distributions used for the privacy schemes (like Gaussian or Cauchy) have infinite support. In real-world digital systems with finite ranges, implementing these exactly is impossible, and the effects of truncation are not explored.
- General Alphabets: While the authors found bounds for Gaussian and discrete variables, finding a tight upper bound for maximal correlation in terms of MI for arbitrary distributions remains an open challenge.
- Practical Implementation: The paper focuses on the "what is possible" (bounds) rather than "how to build it" (algorithms for non-Gaussian data). Future work could involve applying these bounds to specific machine learning privacy techniques like Differential Privacy.
Summary updated