Trust-Based Cloud Machine Learning Model Selection for Industrial IoT and Smart City Services

Basheer Qolomany, Ihab Mohammed, Ala I. Al-Fuqaha, Mohsen Guizani, Junaid Qadir

TL;DR

This paper proposes a trust-based framework for selecting the most reliable machine learning (ML) models from a cloud-based pool to be deployed on resource-constrained IoT devices. By defining "trust" as the consensus among an ensemble of models, the authors developed a polynomial-time heuristic that maximizes system reliability while strictly adhering to communication and reconfiguration budgets.

What Problem Does This Paper Address?

In the era of Machine Learning as a Service (MLaaS), cloud providers often train massive ensembles of models using data collected from the edge. However, deploying these models back to Industrial IoT (IIoT) or Smart City devices presents a three-fold challenge:

  1. Resource Constraints: IoT devices have limited battery, processing power, and intermittent connectivity, meaning they cannot run complex ensembles or switch models constantly.
  2. Security Risks: ML models are vulnerable to "poisoning attacks," where malicious actors inject bad data during training to compromise the model's integrity.
  3. Optimization Complexity: Finding the "perfect" sequence of models to use over time—while minimizing the overhead of switching between them—is an NP-complete problem, making it computationally expensive to solve exactly.

The authors aim to bridge the gap between high-accuracy cloud intelligence and the practical, secure needs of edge devices.

Methods

The core of the paper is the Trusted Machine Learning (TML) selection process.

  • Defining Trust: The authors quantify trust based on the "degree of agreement." If we have five models and Model A’s prediction is very close to the average of the other four, it is considered highly trustworthy. If Model B is an outlier, its trust score drops.
  • The Optimization Model: They first formulate the selection as an Integer Linear Programming (ILP) problem to find the mathematically optimal solution.
  • The Heuristics: Since ILP is too slow for real-time use, they introduce two novel algorithms:
  • Splice Heuristic: This identifies long, consecutive time segments where a single model maintains high trust, effectively "splicing" them together to minimize the need for reconfigurations.
  • Fixing Heuristic: A more advanced approach that uses Linear Programming relaxation to find a "competitive" solution that is nearly as good as the ILP but runs in polynomial time.
  • Fail-Safe Mechanism: If no model in the pool meets a minimum trust threshold, the system triggers a fail-safe alert, informing the device that it should enter a safe state rather than relying on potentially compromised data.

Datasets

The researchers evaluated their approach using two primary proxies for real-world services:

  • Smart City (Traffic Prediction): The CityPulse dataset, which contains six months of vehicular traffic volume data from Aarhus, Denmark.
  • Industrial IoT (Predictive Maintenance): The NASA Turbofan Engine Degradation simulation dataset, used to predict the "Remaining Useful Life" (RUL) of engines.
  • Adversarial Simulation: To test the "trust" metric, they intentionally poisoned 20% of the training data using a "swap percentiles" attack to create malicious models.

Key Insights and Ideas

  • Consensus as a Proxy for Truth: In a decentralized or cloud-to-edge environment, you often don't have the "ground truth" immediately. This paper argues that the "wisdom of the crowd" (ensemble agreement) is a robust way to filter out poisoned models in real-time.
  • The Reconfiguration Budget: The paper highlights that the "best" model isn't always the right choice if it requires a high-bandwidth update every few minutes. The heuristic intelligently balances accuracy against the "cost" of switching models.
  • Fail-Safe over False Confidence: One of the most important "aha" moments is the inclusion of a fail-safe state. In mission-critical infrastructure (like a power grid), knowing that no model is currently trustworthy is more valuable than a guess from a compromised one.

Key Results

  • Near-Optimal Performance: The proposed heuristic performed remarkably close to the "perfect" ILP solution. In the IIoT case study, the trust level was only 0.49% to 3.17% lower than the theoretical optimum. In the Smart City study, it was within 0.7% to 2.53%.
  • Error Reduction: By selecting trusted models, the system achieved significantly lower Root Mean Square Error (RMSE) compared to using individual, potentially poisoned models—reducing error by up to 66.95% in traffic volume predictions.
  • Scalability: Unlike ILP, which struggles as the number of models or time slots increases, the heuristic provides a polynomial-time approximation, making it viable for large-scale cloud deployments.

Limitations and Future Work

The authors acknowledge a few areas for further exploration:

  • Model Diversity: The current study focused primarily on Long Short-Term Memory (LSTM) networks. Future work could test how the heuristic handles a mix of different architectures like CNNs or SVMs.
  • Attack Vectors: The paper specifically targets poisoning attacks. Future iterations could look at other adversarial threats, such as evasion attacks at inference time.
  • Domain Expansion: The authors suggest applying this trust-based selection to other sensitive fields, including medical diagnostics and autonomous vehicle navigation.

Summary updated

BibTeX
@article{qolomany2021trust,
  author = {Basheer Qolomany and Ihab Mohammed and Ala I. Al-Fuqaha and Mohsen Guizani and Junaid Qadir},
  title = {{Trust-Based Cloud Machine Learning Model Selection for Industrial IoT and Smart City Services}},
  journal = {IEEE Internet of Things Journal},
  year = {2021},
  doi = {10.1109/jiot.2020.3022323},
}
Figure