Abstract
The rate-privacy function is defined as a tradeoff between privacy and utility in a distributed private data system in which both privacy and utility are measured using mutual information. Here, we use maximal correlation in lieu of mutual information in the privacy constraint. We first obtain some general properties and bounds for maximal correlation and then modify the rate-privacy function to account for the privacy-constrained estimation problem. We find a bound for the utility in this problem when the maximal correlation privacy is set to some threshold $\epsilon>0$ and construct an explicit privacy scheme which achieves this bound.
Introduction
In the design of distributed data systems, engineers face a fundamental tension: the need to share data for utility while protecting sensitive underlying information. Traditionally, the "Rate-Privacy Function" has relied on Mutual Information (MI) to quantify both the usefulness of shared data and the risk of information leakage. However, MI has notable limitations, particularly in its inability to capture "worst-case" functional dependencies. If a malicious actor can apply a specific function to shared data to perfectly recover a private attribute, MI may not fully reflect that risk.
This research, conducted at Queen’s University, proposes substituting MI with Maximal Correlation in the privacy constraint. By leveraging the unique properties of maximal correlation—such as its robustness against functional transformations and its stability across independent samples—the authors derive new theoretical bounds and an explicit privacy-preserving scheme using $\alpha$-stable noise.
The Case for Maximal Correlation
Maximal correlation ($\rho_m$) is a more stringent measure of dependence than Mutual Information because it seeks the maximum possible correlation between any two functions of the variables. This makes it particularly suited for privacy applications where the attacker's specific goals or methods may be unknown.
One of the most significant advantages of maximal correlation identified in this work is its tensorization property. Unlike Mutual Information, for independent and identically distributed (i.i.d.) data, the maximal correlation of a sequence of samples remains equal to the maximal correlation of a single sample. This property ensures that the privacy metric remains stable and predictable as the scale of the data system grows.
Methodology and Theoretical Framework
The paper establishes its framework through three primary theoretical steps:
- Characterization: The authors define maximal correlation in terms of the singular values of conditional expectation operators, providing a rigorous mathematical foundation for its use as a privacy metric.
- Comparison and Bounds: To bridge the gap between existing MI-based literature and this new approach, the authors derive inequalities relating the two measures. For example, in jointly Gaussian variables, they prove that the square of the maximal correlation is bounded by a constant multiple of the Mutual Information: $\rho_m^2 \leq (2 \ln 2)I(X;Y)$.
- Privacy Modeling: The authors introduce a "privacy filter" model where the shared variable $Z$ is a noisy version of the private variable $X$ ($Z = X + \lambda N$). They utilize $\alpha$-stable distributions—a class that includes Gaussian and Cauchy distributions—to model this noise, allowing for an analytical calculation of the privacy-utility tradeoff.
Key Results: The Privacy-Utility Tradeoff
The core contribution of the paper is the quantification of the "price" of privacy. By measuring utility through the Minimum Mean Squared Error (MMSE), the authors establish a concrete bound for estimation accuracy under a privacy constraint.
The $\alpha$-Stable Noise Scheme
The authors demonstrate that using noise from an $\alpha$-stable distribution allows for a precise "privacy knob" ($\lambda$). They provide a closed-form expression for the resulting privacy level: $$\rho_m = \frac{1}{\sqrt{1 + \lambda^\alpha}}$$ This shows that as the noise scale $\lambda$ increases, the maximal correlation (and thus the information leakage) decreases predictably.
The Fundamental Bound
The paper's main result (Theorem 6) derives a lower bound for the MMSE of estimating a utility variable $Y$ from shared data $Z$, given a privacy threshold $\epsilon$: $$mmse_\epsilon(Y) \geq (1 - \varrho_\epsilon^2(X; Y))var(Y)$$ This formula provides a theoretical limit on utility: as the privacy constraint is tightened (decreasing $\epsilon$), the minimum possible estimation error for the utility variable $Y$ necessarily increases. This gives system designers a mathematical ceiling for the performance of any privacy-preserving estimation system.
Limitations and Future Directions
While the paper provides a robust theoretical foundation, several practical challenges remain:
- Infinite Support: The $\alpha$-stable distributions used in the proposed schemes have infinite support. Implementing these in digital systems with finite ranges requires truncation, the effects of which are not yet fully explored.
- General Alphabets: While bounds were established for Gaussian and discrete variables, finding tight upper bounds for maximal correlation in terms of MI for arbitrary distributions remains an open mathematical challenge.
- Algorithmic Implementation: The current work focuses on theoretical bounds. Future research could bridge this theory with practical machine learning techniques, such as integrating maximal correlation constraints into Differential Privacy frameworks.
Summary updated