Distributed One-class Learning

Ali Shahin Shamsabadi, Hamed Haddadi, Andrea Cavallaro

Introduction

The unauthorized sharing of privacy-sensitive images, such as "revenge porn" or non-consensual photography, has become a significant crisis on social media platforms. Traditionally, content filtering requires service providers to access sensitive images to create hashes or train classifiers, creating a privacy paradox: users must surrender their privacy to a central authority to protect it.

Existing distributed learning methods like Federated Learning attempt to mitigate this by training models locally. However, they remain vulnerable to information leakage—where adversaries reverse-engineer training data from model parameters—and struggle with class imbalance when a user only possesses data for their own "class" (e.g., their own face). To address these challenges, researchers have developed Distributed One-Class Learning (DOCL), a cloud-based filtering system that decomposes a global filter into individual one-class autoencoders trained locally on edge devices.

Methodology

The DOCL architecture shifts from a monolithic multi-class model to a collection of $N$ independent one-class classifiers. The process involves three main stages:

  1. Local Training: Each user $u_i$ utilizes a pre-trained ResNet to extract features from their private images. They then train a local autoencoder to compress and reconstruct these specific features. Because the model only observes the user's data, it becomes highly specialized.
  2. Parameter Upload: Users upload only their autoencoder parameters and the statistical distribution (mean $\mu_i$ and standard deviation $\sigma_i$) of their reconstruction errors to the cloud. These parameters are never shared with other users, preventing the primary leakage vectors found in standard federated learning.
  3. Global Filtering and the Guard Module: When a third party attempts to upload an image $I^t$, the cloud passes it through all $N$ stored autoencoders. The system calculates the reconstruction error for each. If the minimum error falls within a "privacy interval" $[\mu_i - \alpha\sigma_i, \mu_i + \alpha\sigma_i]$ for any user $i$, the "Guard" module identifies the image as sensitive and blocks the upload.
Architecture of the DOCL system showing the flow from feature extraction to the Guard module.

Experimental Setup

The system was evaluated using three distinct datasets to simulate various content types:

  • IMDB: 10 celebrity face sets representing the core task of identity protection.
  • CIFAR-10: 10 classes of general objects (e.g., airplanes, cats) to test performance against high inter-class variability.
  • MNIST: 10 classes of handwritten digits for structured data testing.
  • Fashion-MNIST: Used as "non-privacy-sensitive" data to evaluate the "acceptance rate" (the frequency with which safe images are correctly allowed).

Key Results

DOCL demonstrated high efficacy across different data types. On the IMDB dataset, the system achieved between 92% and 97% accuracy in blocking unauthorized face uploads. Performance on the MNIST dataset was similarly robust, reaching 97% overall accuracy.

Per-class accuracy across ten users, demonstrating the effectiveness of the one-class classification approach.

The system's performance is sensitive to the $\alpha$ parameter in the Guard module. Increasing $\alpha$ improves the detection of private images (higher accuracy) but may increase the false positive rate, accidentally blocking "safe" images. Furthermore, the framework proved robust against adversarial attempts to "poison" autoencoders, particularly on complex data like faces.

Discussion and Insights

The core innovation of DOCL lies in isolation as security. By ensuring that one user's parameters never influence another's local model, the system breaks the feedback loops required for membership inference attacks. Furthermore, one-class learning is uniquely suited for privacy contexts where every user is their own "class"; unlike traditional classifiers, these models do not require negative samples to define boundaries.

The "reconstruction gap"—the high error produced when an autoencoder trained on User A attempts to reconstruct User B's data—serves as a natural, privacy-preserving distance metric. Additionally, the system is highly scalable; adding a new user is an $O(1)$ operation for the existing network, requiring only the addition of a new module without retraining.

Limitations and Future Work

While effective, DOCL faces challenges regarding cloud computational costs. As the user base $N$ grows, the cloud must execute $N$ autoencoders for every upload. Although autoencoders are computationally light, this could become a bottleneck at a massive scale. Additionally, performance was lower on CIFAR-10 (~80%) due to high intra-class variability (e.g., the diversity within the "bird" category), leading to reconstruction overlaps. Future research will explore extending this framework to video and audio data and testing on larger, more diverse datasets.

Summary updated

Abstract

We propose a cloud-based filter trained to block third parties from uploading privacy-sensitive images of others to online social media. The proposed filter uses Distributed One-Class Learning, which decomposes the cloud-based filter into multiple one-class classifiers. Each one-class classifier captures the properties of a class of privacy-sensitive images with an autoencoder. The multi-class filter is then reconstructed by combining the parameters of the one-class autoencoders. The training takes place on edge devices (e.g. smartphones) and therefore users do not need to upload their private and/or sensitive images to the cloud. A major advantage of the proposed filter over existing distributed learning approaches is that users cannot access, even indirectly, the parameters of other users. Moreover, the filter can cope with the imbalanced and complex distribution of the image content and the independent probability of addition of new users. We evaluate the performance of the proposed distributed filter using the exemplar task of blocking a user from sharing privacy-sensitive images of other users. In particular, we validate the behavior of the proposed multi-class filter with non-privacy-sensitive images, the accuracy when the number of classes increases, and the robustness to attacks when an adversary user has access to privacy-sensitive images of other users.
BibTeX
@inproceedings{shamsabadi2018distributed,
  author = {Ali Shahin Shamsabadi and Hamed Haddadi and Andrea Cavallaro},
  title = {{Distributed One-class Learning}},
  booktitle = {ICIP},
  year = {2018},
  url = {https://openreview.net/forum?id=8KdkrtY8bRF},
}
Figure