Frequency Oracle for Sensitive Data Monitoring (Student Abstract)

Richard Sances, Olivera Kotevska, Paul Laiu

Introduction

As IoT devices and health monitors become ubiquitous, they collect vast amounts of sensitive personal data. Protecting user privacy while extracting meaningful insights is a critical challenge. Local Differential Privacy (LDP) has emerged as a robust solution, where data is "privatized" (noised) on the user's device before being transmitted to a central server.

A fundamental component of LDP is the Frequency Oracle (FO), a protocol designed to estimate the frequency of specific values within a dataset. While numerous FO algorithms exist—ranging from hash-based methods to Hadamard transforms—there is currently limited guidance on which algorithm is best suited for specific data scales or privacy requirements. This research provides a practical roadmap for selecting the optimal FO based on empirical benchmarking against real-world energy consumption data.

Methodology and Taxonomy

The researchers categorized existing FO algorithms into a structured taxonomy based on their underlying mathematical mechanisms. This classification helps in understanding the diverse landscape of privacy-preserving protocols.

Taxonomy of Frequency Oracle algorithms categorized by mechanism.

The study evaluated these algorithms by varying three primary factors: data size, the presence of frequency peaks (spikes in data), and the privacy budget ($\\epsilon$). Additionally, the algorithms were tested within a Federated Learning (FL) framework to compare the accuracy of models when privatization was applied to raw client data versus model weights.

Experimental Setup

The benchmarking utilized two significant real-world time-series datasets:

  1. Smart Meters in London (UKPowerNetworks 2018): Energy data from 5,567 households (2012–2014).
  2. Energy Consumption in Germany (Milojkovic 2021): Data from 955 households (2018–2022).

Data was preprocessed by assigning unique integer identifiers to users and splitting the sets into 70% training and 30% testing for Federated Learning tasks.

Key Results and Analysis

The Impact of Data Scale

The most significant finding of this research is that the "optimal" algorithm is highly dependent on the scale of the population being monitored. There is no universal Frequency Oracle; rather, the choice must be tailored to the dataset size.

Mean Square Error comparison showing how Direct Encoding, Unary Encoding, and RAPPOR perform at different data scales.

Based on the Mean Square Error (MSE) analysis, the researchers established a clear hierarchy:

  • Small Datasets: Direct Encoding provides the highest accuracy.
  • Large Datasets: Unary Encoding becomes the superior choice.
  • Massive Datasets: RAPPOR (Randomized Aggregatable Privacy-Preserving Ordinal Response) outperforms all other methods.

Privacy Budget and Robustness

While most algorithms showed improved accuracy as the privacy constraint loosened (higher $\\epsilon$), RAPPOR demonstrated unique stability, maintaining consistent performance even with tighter privacy budgets. Furthermore, the presence of "peaks" or non-uniform data distributions had a negligible effect on performance, suggesting these oracles are robust against real-world data irregularities.

Performance in Federated Learning

In the context of Federated Learning, the study found that privatizing model weights (parameters) resulted in significantly lower error rates than privatizing raw data at the client level. Algorithms such as Unary Encoding, Histogram Encoding, Fast Local Hashing, and Hadamard Response maintained high prediction accuracy in FL settings. Conversely, Count Mean Sketch and the Hadamard Mechanism struggled to produce reliable predictions when integrated into the FL framework.

Conclusion and Future Work

This study serves as a foundational benchmark for implementing Local Differential Privacy in sensitive monitoring applications. The results indicate that while data size and privacy budgets are critical factors, the range of data values has a minimal impact on algorithm accuracy.

Future research will focus on expanding the library of tested algorithms and developing an automated recommendation tool. This tool will analyze specific data properties and privacy requirements to automatically select the optimal Frequency Oracle, streamlining the implementation of LDP for developers and researchers alike.

Summary updated

Abstract

As data privacy issues grow, finding the best privacy preservation algorithm for each situation is increasingly essential. This research has focused on understanding the frequency oracles (FO) privacy preservation algorithms. FO conduct the frequency estimation of any value in the domain. The aim is to explore how each can be best used and recommend which one to use with which data type. We experimented with different data scenarios and federated learning settings. Results showed clear guidance on when to use a specific algorithm.
BibTeX
@inproceedings{sances2024frequency,
  author = {Richard Sances and Olivera Kotevska and Paul Laiu},
  title = {{Frequency Oracle for Sensitive Data Monitoring (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30507},
}
Figure