Privacy-Preserving Similarity Metrics for Distributed Data

This direction is split into other directions: privacy-preserving similarity estimation. Its last version stays readable and citable here.

Version 1 of 164 papers68 conceptsfrozen 2026-10-04

In a centralized world, measuring the similarity between two datasets is trivial: you simply compare their distributions using metrics like Kullback-Leibler divergence or the Fréchet distance. However, in decentralized environments like Federated Learning (FL), data is siloed across devices or institutions that cannot share raw information due to privacy regulations or proprietary concerns.

This creates a "blind comparison" problem. To build effective collaborative models, a central server needs to know which clients are similar enough to group together, yet it is forbidden from seeing the very data required to make that determination. Researchers have developed a suite of techniques to estimate these distances through proxies, statistical summaries, and cryptographic protocols.

Model Updates as Data Proxies

The most intuitive way to measure data similarity without seeing the data is to look at the "behavior" of models trained on that data. In federated learning, a client’s model update (its gradient) represents the direction the local data "wants" the global model to move.

If two clients consistently produce similar gradients, it is a strong signal that their underlying data distributions are aligned. Hyeji Kim leverages this in CFL-GP, using spectral clustering on accumulated gradient features to group clients 100x faster than traditional methods. Similarly, Thanveer Shaik and colleagues in Clustered FedStack use the weights of the model's output layer as a proxy, applying the Bayesian Information Criterion to automatically determine the optimal number of client clusters.

Statistical Fingerprinting

Beyond gradients, researchers use lightweight statistical summaries—"fingerprints"—that capture the essence of a dataset without revealing individual samples.

For instance, Patrick Godau and A. Srivastava introduced Task Fingerprinting for medical imaging. By aggregating feature activations into binned histograms, they can calculate the similarity between diverse medical tasks (like MRI vs. CT scans) using a binned KL-divergence. This allows institutions to "borrow" successful training configurations from similar tasks without ever exchanging patient images.

In the healthcare space, Yiqiang Chen and Lu Wang demonstrate in FedAP that even simple batch normalization statistics can serve as a powerful similarity metric, allowing for smarter model aggregation that outperforms standard FL by over 10%.

Geometric and Latent Distances

Some of the most robust similarity measures come from the field of optimal transport. The Wasserstein distance provides a principled way to measure the "work" required to transform one distribution into another.

In Federated Wasserstein Distance, the authors (including Alain Rakotomamonjy) use the geometric properties of geodesics to estimate this distance across private clients. This allows a server to orchestrate distance calculations without seeing raw samples.

Alternatively, Achintha Wijesinghe and Zhi Ding 0001 propose PFL-GAN, which uses Generative Adversarial Networks to map data into a latent feature space. Instead of sharing data, clients share local GAN models, which the server uses to measure similarity and generate synthetic datasets for personalized training.

Cryptographic Safeguards

When high-precision similarity is required, researchers turn to heavy-duty cryptographic tools like Fully Homomorphic Encryption (FHE) and Differential Privacy (DP).

Yuping Yan and Yaochu Jin developed FHE-FCDF, a method that quantifies how "non-IID" (different) a client's data is by comparing Cumulative Distribution Functions (CDFs) within an encrypted domain. This ensures that neither the server nor other participants can see the local statistics.

For comparing private user data against public datasets, Charlie Hou and Giulia Fanti introduced FreD (Federated Private Fréchet Distance). As described in Privately Customizing Prefinetuning, FreD uses DP-noisy mean and covariance estimates combined with secure aggregation to help developers select the best public pre-training data for a specific private user base.

Why Similarity Matters

The ability to measure distance privately is not just a theoretical exercise; it is the engine behind several critical advancements in decentralized learning:

  • Optimal Collaboration: FedCollab by Wenxuan Bao and Jingrui He uses distribution distance to cluster clients into "coalitions," preventing "negative transfer" where a global model performs worse than a local one.
  • Fairness and Overlap: In graph-based networks, Zihao Zhou and Xuebin Ren use privacy-preserving estimation in FairGFL to detect overlapping subgraphs, ensuring that model performance remains consistent across imbalanced participants.
  • Strategic Unlearning: When a user exercises their "right to be forgotten," Jiaqi Shao and Qiang Yang show in Beyond Right to be Forgotten that similarity metrics are essential to identify which remaining clients will be most affected by the data withdrawal, allowing for incentive mechanisms to stabilize the network.

Papers in this version

Showing 60 of 64 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_privacy_preserving_similarity_metrics_fo_c50ca13b,
  title = {Privacy-Preserving Similarity Metrics for Distributed Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/privacy-preserving-similarity-metrics-for-distributed-data/rdr_c50ca13bc25d444da2b78baa309b9200},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_privacy_preserving_similarity_metrics_fo_c50ca13b_v1,
  title = {Privacy-Preserving Similarity Metrics for Distributed Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/privacy-preserving-similarity-metrics-for-distributed-data/rdr_c50ca13bc25d444da2b78baa309b9200/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}