Distributed One-Class Learning

Ali Shahin Shamsabadi, Hamed Haddadi, Andrea Cavallaro

TL;DR

Researchers have developed DOCL (Distributed One-Class Learning), a cloud-based filtering system designed to block the unauthorized upload of privacy-sensitive images. By decomposing a global filter into individual one-class autoencoders trained locally on edge devices, the system prevents the cloud provider—and other users—from ever seeing a user's private images or model parameters.

What Problem Does This Paper Address?

Unauthorized sharing of privacy-sensitive images (such as "revenge porn" or photos of others taken without consent) is a growing crisis on social media. Traditionally, to filter such content, a service provider would need access to the sensitive images themselves to create "hashes" or train a classifier. This creates a paradox: to protect your privacy, you must first surrender it to a central authority.

Existing distributed learning methods (like Federated Learning) attempt to solve this by training models locally and only sharing "gradients" or parameters. However, these methods have two major flaws:

  1. Information Leakage: Adversaries can often reverse-engineer training data by analyzing changes in the global model parameters.
  2. Class Imbalance: Standard multi-class classifiers struggle when one user only has data for their own "class" (their own face), making the system hard to scale as new users join.

Methods

The authors propose DOCL, which shifts the architecture from one giant multi-class model to a collection of $N$ independent one-class classifiers.

  1. Local Training (The Edge): Each user $u_i$ uses a pre-trained ResNet to extract features from their private images. They then train a local autoencoder—a neural network designed to compress an image and then reconstruct it perfectly. Because the autoencoder only sees the user’s specific images, it becomes an expert at reconstructing only that user's data.
  2. Parameter Upload: Users upload only their autoencoder parameters and the distribution (mean and standard deviation) of their reconstruction errors to the cloud. Crucially, these parameters are never shared back down to other users, eliminating the primary leakage vector in federated learning.
  3. The Global Filter (The Cloud): When a third party tries to upload a new image $I^t$, the cloud passes it through all $N$ stored autoencoders.
  4. The Guard Module: The system calculates the reconstruction error for each autoencoder. If the minimum error falls within a specific "privacy interval" $[\mu_i - \alpha\sigma_i, \mu_i + \alpha\sigma_i]$ for any user $i$, the "Guard" identifies the image as privacy-sensitive and blocks the upload.

Datasets

The researchers evaluated DOCL using three distinct datasets to simulate different types of image content:

  • IMDB: 10 celebrity face sets used to represent the core task of protecting personal identity.
  • CIFAR-10: 10 classes of general objects (airplanes, cats, etc.) to test performance against high inter-class variability.
  • MNIST: 10 classes of handwritten digits to test the system on simpler, highly structured data.
  • Fashion-MNIST: Used as "non-privacy-sensitive" data to test the "acceptance rate" (how often the filter correctly allows safe images through).

Key Insights and Ideas

  • Isolation as Security: By ensuring that user $A$'s parameters are never used to update user $B$'s local model, the authors break the feedback loop that usually allows for "membership inference" attacks.
  • One-Class vs. Multi-Class: In a world where every user is their own "class," traditional classifiers fail because they need to see examples of what they aren't (negative samples). One-class learning thrives here because it only needs to know what the target looks like.
  • The Reconstruction Gap: The "aha moment" is that an autoencoder trained on "User A" will produce a high error when trying to reconstruct "User B." This error serves as a natural, privacy-preserving distance metric.
  • Scalability: Adding a new user is "O(1)" for the existing network—you simply plug in a new one-class module without retraining the rest of the system.

Key Results

  • High Accuracy on Faces: On the IMDB dataset, the system achieved between 92% and 97% accuracy in blocking unauthorized face uploads, depending on the size of the autoencoder's representation layer.
  • Superiority on MNIST: The system reached 97% overall accuracy on digit recognition.
  • The Confidence Trade-off: By adjusting the $\alpha$ parameter in the Guard module, providers can tune the system. A higher $\alpha$ catches more private images (higher accuracy) but might accidentally block more "safe" images (lower acceptance rate).
  • Robustness to Adversaries: The paper demonstrates that even if an adversary gains access to a victim's images and tries to train a "poisoned" autoencoder to fool the system, the DOCL framework remains robust, especially on complex data like faces.

Limitations and Future Work

  • Cloud Computational Cost: As the number of users $N$ grows, the cloud must run $N$ autoencoders for every single upload. While autoencoders are computationally "light," this could become a bottleneck at the scale of millions of users.
  • Intra-class Variability: Performance was lower on CIFAR-10 (80%) because objects like "birds" or "dogs" have massive internal variety compared to a specific person's face, leading to more reconstruction overlaps.
  • Scope: The current work focuses on images. The authors suggest that future work should extend this to other sensitive data types (like video or audio) and test on even larger, more diverse datasets.

Summary updated

Abstract

We propose a cloud-based filter trained to block third parties from uploading privacy-sensitive images of others to online social media. The proposed filter uses Distributed One-Class Learning, which decomposes the cloud-based filter into multiple one-class classifiers. Each one-class classifier captures the properties of a class of privacy-sensitive images with an autoencoder. The multi-class filter is then reconstructed by combining the parameters of the one-class autoen-coders. The training takes place on edge devices (e.g. smartphones) and therefore users do not need to upload their private and/or sensitive images to the cloud. A major advantage of the proposed filter over existing distributed learning approaches is that users cannot access, even indirectly, the parameters of other users. Moreover, the filter can cope with the imbalanced and complex distribution of the image content and the independent probability of addition of new users. We evaluate the performance of the proposed distributed filter using the exemplar task of blocking a user from sharing privacy-sensitive images of other users. In particular, we validate the behavior of the proposed multi-class filter with non-privacy-sensitive images, the accuracy when the number of classes increases, and the robustness to attacks when an adversary user has access to privacy-sensitive images of other users.
BibTeX
@inproceedings{shamsabadi2018distributed,
  author = {Ali Shahin Shamsabadi and Hamed Haddadi and Andrea Cavallaro},
  title = {{Distributed One-Class Learning}},
  booktitle = {ICIP},
  year = {2018},
  url = {https://openreview.net/forum?id=8KdkrtY8bRF},
}

Loading related content...

Figure