SODA: Protecting Proprietary Information in On-Device Machine Learning Models

Akanksha Atrey, Ritwik Sinha, Saayan Mitra, Prashant J. Shenoy

SODA: Keeping the "Secret Sauce" of On-Device AI Under Lock and Key

TL;DR

As machine learning (ML) moves from the cloud to our pockets, service providers face a new risk: users can "interrogate" on-device models to steal proprietary business logic. SODA (Secure On-Device Application) is a new framework that detects these adversarial queries with 89% accuracy in under 50 attempts, protecting a company’s intellectual property without ruining the user experience.

What Problem Does This Paper Address?

Traditionally, ML models live in the cloud, protected by a "security perimeter." When you ask for a credit card recommendation, the cloud processes your data and sends back a result. However, to improve privacy and reduce latency, many companies now deploy these models directly onto user devices (phones, IoT gadgets).

The problem? Once a model is on a user's device, it’s outside the provider's control. A "curious" user or a competitor can query the model thousands of times to reverse-engineer its "proprietary information"—the specific rules, offer distributions, or decision boundaries that give a company its competitive edge. While previous research focused on "model stealing" (copying the whole model), this paper highlights that stealing the logic inside is much easier, faster, and just as damaging.

Methods

The authors propose SODA, an end-to-end framework designed to detect adversarial usage patterns over time. Its architecture relies on three main pillars:

  1. The Autoencoder (AE) Foundation: Instead of feeding raw data directly into the service model, SODA uses an autoencoder. This compresses the input into a "latent representation." This serves two purposes: it makes the service model more efficient and provides a baseline for "normal" data.
  2. The Leakage Rate ($l_t$): This is SODA’s "heartbeat" detector. It calculates a score based on three metrics:
  3. Reconstruction Error: Does the input look like the training data? (Detects random/garbage queries).
  4. Query Distance: Are the queries too similar (probing a boundary) or too different (trying to map the whole output space)?
  5. Output Entropy: Is the model being forced to produce an unusually diverse or repetitive set of answers?
  6. Secure Deployment: SODA uses ONNX for model interoperability and XSalsa20 stream ciphers to encrypt model files on the device, ensuring that even if a user explores their phone's file system, the model remains a black box.

Datasets

The researchers evaluated SODA using two widely recognized benchmarks:

  • UCI Human Activity Recognition (HAR): 561 smartphone sensor features used to predict activities like walking or sitting.
  • MNIST: The classic dataset of 70,000 handwritten digits (0-9).

These were used to simulate "organic" (benign) user behavior versus adversarial attempts to map out every possible model response.

Key Insights and Ideas

  • Proprietary Info vs. Model Stealing: The authors argue that you don't need to clone a model to exploit it. Learning the "rules" (e.g., "What specific sensor data triggers a high-value insurance offer?") is a much lower bar for an attacker but carries high business risk.
  • The Robustness of Random Forests: Interestingly, the study found that Random Forests (RF) are naturally more resistant to these attacks than Deep Neural Networks (DNNs) because their decision boundaries are "unsmooth" and harder to probe with simple noise.
  • Efficiency through Compression: By using an autoencoder to reduce input dimensionality, SODA actually speeds up inference for Random Forest models (a ~0.16-fold decrease in runtime), proving that security doesn't always have to come at a performance cost.

Key Results

  • High Detection Accuracy: SODA detected adversarial usage with 89% accuracy on the HAR dataset and up to 94.9% on MNIST.
  • Speed of Detection: Unlike prior methods that might need hundreds of queries to flag a user, SODA identifies threats in less than 50 queries.
  • Superiority over Baselines: SODA outperformed existing defenses like MagNet and PRADA by 10% to 59% in accuracy, largely because it looks at multiple dimensions of "weirdness" (distance, error, and entropy) rather than just one.
  • Minimal Overhead: The system adds negligible latency (averaging 0.158 seconds for the detector) and can run concurrently with the main service.

Limitations and Future Work

While SODA is highly effective against individual "curious" users, the authors acknowledge a few hurdles:

  • Colluding Adversaries: If multiple users coordinate their queries and share results, they might stay under the detection threshold of a single device.
  • Generative AI: The current work focuses on classification models. Protecting the proprietary logic within Generative AI (like LLMs) on-device is a significantly more complex challenge.
  • Library Dependencies: The current prototype relies on the scipy library for some calculations, which increases the application size. Future versions could be optimized with custom mathematical implementations to further shrink the footprint for low-end IoT devices.

Summary updated

Abstract

The growth of low-end hardware has led to a proliferation of machine learning-based services in edge applications. These applications gather contextual information about users and provide some services, such as personalized offers, through a machine learning (ML) model. A growing practice has been to deploy such ML models on the user's device to reduce latency, maintain user privacy, and minimize continuous reliance on a centralized source. However, deploying ML models on the user's edge device can leak proprietary information about the service provider. In this work, we investigate on-device ML models that are used to provide mobile services and demonstrate how simple attacks can leak proprietary information of the service provider. We show that different adversaries can easily exploit such models to maximize their profit and accomplish content theft. Motivated by the need to thwart such attacks, we present an end-to-end framework, SODA, for deploying and serving on edge devices while defending against adversarial usage. Our results demonstrate that SODA can detect adversarial usage with 89% accuracy in less than 50 queries with minimal impact on service performance, latency, and storage.
BibTeX
@inproceedings{atrey2023soda,
  author = {Akanksha Atrey and Ritwik Sinha and Saayan Mitra and Prashant J. Shenoy},
  title = {{SODA: Protecting Proprietary Information in On-Device Machine Learning Models}},
  booktitle = {SEC},
  year = {2023},
  url = {https://openreview.net/forum?id=tfuly949GO},
}
Figure