Quantifying Privacy Leakage with Information Theory

605 concepts387 papers1 proposal

When sharing data or deploying machine learning models, we face a fundamental dilemma: how do we maximize utility while preventing adversaries from inferring sensitive attributes? While frameworks like Differential Privacy (DP) provide rigorous guarantees for database membership, they can be overly conservative or misaligned when the goal is to prevent the reconstruction of specific features, states, or continuous variables.

Information-theoretic privacy metrics offer an alternative paradigm. By treating data release as a noisy communication channel, this approach directly measures and bounds the statistical dependence—quantified in bits—between private data and released outputs.

The Intuition: Mutual Information as an Adversarial Bound

To understand why information theory provides a robust foundation for privacy, consider a private variable $X$ (such as raw user data, locations, or household energy consumption) and a released variable $Y$ (such as a trained model, an aggregate statistic, or a compressed representation). The core metric is Mutual Information (MI), denoted as $I(X; Y)$, which measures the reduction in uncertainty about $X$ after observing $Y$. If $I(X; Y) = 0$, the variables are statistically independent, meaning an adversary can infer nothing about $X$ from $Y$ better than random guessing based on prior knowledge.

The operational power of this metric stems from its direct connection to estimation theory. By bounding the mutual information between the input dataset and the output, we place a mathematical ceiling on an adversary's inference accuracy. As John Duchi et al. demonstrate in Privacy Aware Learning, a worst-case mutual information metric can define formal privacy budgets that remain entirely independent of specific noise distributions or algorithmic implementations.

This mathematical bound translates directly into protection against real-world attacks:

The Privacy-Utility Trade-off via the Information Bottleneck

How do we optimize a system to be simultaneously useful and private? This challenge maps naturally to rate-distortion theory and the Information Bottleneck principle. Suppose we want to compress or transform raw data $X$ into a representation $Y$ that preserves a public task label $T$ (utility) while scrubbing a sensitive attribute $S$ (privacy). The objective is to maximize $I(Y; T)$ while minimizing $I(Y; S)$.

As Amy Zhang 0001 et al. demonstrate in Managing your Private and Public Data: Bringing down Inference Attacks against your Privacy, this balance can be formalized as a convex optimization problem, proving that minimizing the mutual information between public and private attributes provides a formal guarantee on the maximum information an adversary can extract.

To operationalize this trade-off in deep learning architectures, Homa Esfahanizadeh et al. introduce InfoShape: Task-Based Neural Data Shaping via Mutual Information. They employ a Lagrangian optimization framework that combines the competing mutual information terms into a single objective function:

$$\mathcal{L} = -I(Y; T) + \beta I(Y; S)$$

Here, the hyperparameter $\beta$ acts as a dial, allowing practitioners to precisely tune the system between maximum utility ($\beta \to 0$) and absolute privacy ($\beta \to \infty$). Similarly, Sayedeh Leila Noorbakhsh et al. leverage this multi-objective optimization in Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attacks to learn representations that simultaneously defend against membership, property, and data reconstruction attacks.

Overcoming the Computational Bottleneck

While the theory behind mutual information is elegant, calculating it exactly in high-dimensional spaces—such as deep neural networks or complex graphs—is notoriously difficult because it requires knowing the underlying probability distributions. To make these metrics practical, researchers use differentiable upper bounds and statistical proxies.

For graph neural networks, Binghui Wang et al. introduce a contrastive log-ratio upper bound in Privacy-Preserving Representation Learning on Graphs: A Mutual Information Perspective. This bound serves as a differentiable penalty during training, systematically squeezing the information shared between graph embeddings and private links until the adversary's inference performance drops to the level of random guessing.

Alternatively, instead of estimating mutual information directly through expensive neural estimators, Haoyue Bai 0002 et al. utilize the Hilbert-Schmidt Independence Criterion (HSIC) in Privacy-preserving data reprogramming. HSIC serves as a computationally efficient proxy for mutual information, measuring the statistical dependence between generated features and sensitive labels directly from data samples. This eliminates the need to train adversarial networks during the optimization loop, drastically accelerating training.

Sequential Decision-Making and Dynamic Systems

Privacy leakage is rarely a one-time event; in sequential systems like smart grids or reinforcement learning (RL) agents, information leaks continuously over time. Information theory handles this by evaluating the mutual information rate across sequences.

In smart grid infrastructures, Deniz Gunduz et al. model privacy in Smart Meter Privacy for Multiple Users in the Presence of an Alternative Energy Source by calculating the mutual information rate between a household's actual energy consumption and the high-frequency readings reported to the utility provider. By treating a physical battery or solar panel as an information-theoretic "privacy filter," they solve for the optimal energy management policy that minimizes this rate.

For active agents executing tasks, Chris Cundy et al. develop a framework in Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients that regularizes the mutual information between an agent's actions and protected state variables. This allows the agent to learn policies that actively mask its private state (e.g., altering behavior to hide its location) while still achieving its primary goal. To accommodate human variability in these loops, Mojtaba Taherisadr et al. propose adaPARL: Adaptive Privacy-Aware Reinforcement Learning for Sequential-Decision Making Human-in-the-Loop Systems, which dynamically tracks the statistical dependence between system actions and user states, reducing predictability by nearly half without sacrificing system utility.

Conditional Leakage and Contextual Privacy

In many practical scenarios, an adversary is not starting from scratch; they already possess side information, or the system architecture dictates a specific flow of information. To isolate the incremental leakage caused by a specific data release, researchers turn to Conditional Mutual Information (CMI), denoted as $I(X; Y | Z)$, which measures the information shared between $X$ and $Y$ given that $Z$ is already known.

This metric is vital for auditing modern bottleneck architectures and defending against targeted attacks:

Ultimately, conditional information metrics allow for the formalization of flexible, context-aware privacy frameworks. As Theshani Nuradha et al. demonstrate in Pufferfish Privacy: An Information-Theoretic Study, bounding the conditional mutual information between a mechanism's output and a protected secret given public information allows researchers to operationalize the Pufferfish privacy framework. This mathematical bridge simplifies privacy auditing in high-dimensional settings and ensures that guarantees can be rigorously composed across multiple queries.

Taking this a step further for data curation, Suraj Nandkishor Kothawade demonstrates in PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Data Subset Selection that submodular conditional mutual information functions can be optimized to build data selection pipelines that simultaneously maximize relevance to a target task while strictly excluding sensitive or private data slices.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_information_theoretic_privacy_metrics_0071722e,
  title = {information-theoretic privacy metrics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/information-theoretic-privacy-metrics/rdr_0071722ee2904959afcf18ae60d529c7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_information_theoretic_privacy_metrics_0071722e_v1,
  title = {information-theoretic privacy metrics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/information-theoretic-privacy-metrics/rdr_0071722ee2904959afcf18ae60d529c7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake