Federated Learning for Autoencoder-based Condition Monitoring in the Industrial Internet of Things

Soeren Becker, Kevin Styp-Rekowski, Oliver Vincent Leon Stoll, Odej Kao

TL;DR

This paper introduces a privacy-preserving, bandwidth-efficient Federated Learning (FL) framework for monitoring the health of industrial machinery. By deploying LSTM-Autoencoders directly to edge devices, the authors achieve state-of-the-art anomaly detection while reducing network traffic by over 99% compared to traditional cloud-based approaches.

What Problem Does This Paper Address?

In the era of Industry 4.0, "Condition Monitoring" (CM) is essential for predicting equipment failure before it causes costly downtime. Traditionally, this requires streaming massive amounts of high-frequency vibration data from sensors to a centralized cloud server for analysis.

However, this approach hits three major walls:

  1. Bandwidth & Latency: Remote industrial sites often rely on unreliable or expensive connections (like 4G or satellite), making the transfer of gigabytes of raw sensor data impractical.
  2. Data Privacy: Raw sensor data can reveal proprietary business information or production secrets, making companies hesitant to share it with third-party cloud providers.
  3. Resource Constraints: While edge devices (like Raspberry Pis) are becoming more powerful, they still lack the memory and processing power to run standard deep learning training pipelines designed for high-end servers.

Methods

The authors propose a Federated Learning (FL) architecture using an LSTM-Autoencoder (AE).

  • The Model: The Autoencoder is trained only on "normal" operational data. It learns to compress and then reconstruct the vibration signal. If a machine starts to fail, the signal deviates from the "normal" pattern, the reconstruction error spikes, and an anomaly is detected.
  • The FL Process: Instead of sending data to the cloud, the "Training Nodes" (edge devices) train the model locally. They only send the model weights (mathematical parameters) to a central "Aggregation Node." The cloud server averages these weights using the FedAvg algorithm and sends a refined "global model" back to the devices.
  • Edge Optimization: To make this work on low-power hardware, the authors modified the standard approach. Instead of processing multiple preceding batches of data (which is memory-intensive), they split single batches into smaller "windows." This allows the model to capture temporal dependencies without exhausting the edge device's RAM.

Datasets

The researchers evaluated their approach using two primary sources:

  1. NASA IMS Bearing Dataset: A well-known public benchmark containing high-frequency vibration data from bearings run until failure.
  2. Rotating Machine (RM) Dataset: A real-world dataset provided by an industry partner, featuring vibration metrics from five different types of rotating machinery (pumps, motors, etc.) with labeled anomalies.

Key Insights and Ideas

  • Knowledge Sharing Without Data Sharing: FL allows different industrial sites—or even different companies—to benefit from a collective model that has "seen" various failure modes without any participant ever seeing another's raw data.
  • The "Cold-Start" Solution: The authors demonstrated that a model can be trained incrementally as new data arrives. This is vital for new factories that don't have years of historical data to train on.
  • Efficiency is the Enabler: By moving from a multi-batch to a windowed single-batch approach, the authors proved that deep learning training is no longer "cloud-only." It can happen on a $50 Raspberry Pi sitting right next to the machine.

Key Results

  • Massive Bandwidth Savings: The FL approach reduced network utilization by up to 99.22% compared to sending raw data. For a remote site, this is the difference between a feasible system and an impossible one.
  • Competitive Accuracy: Despite the distributed nature and hardware constraints, the FL model achieved an average F1-score of 99.4% on the industry dataset, matching the performance of centralized "state-of-the-art" baselines.
  • Reduced Memory Footprint: The proposed method required only 2-3GB of RAM for training, whereas the centralized baseline required 9-11GB, making it compatible with standard edge gateways.

Limitations and Future Work

While the results are promising, the authors acknowledge a few areas for growth:

  • Non-IID Data: The study assumed that data across different sites is relatively similar (Independent and Identically Distributed). In reality, different machines might have different "normal" baselines due to age or environment.
  • Aggregation Algorithms: The paper uses FedAvg, the "standard" FL aggregation method. The authors plan to test more sophisticated weighting algorithms that might handle data heterogeneity better.
  • Broader Hardware Testing: Future work will involve testing on an even wider variety of edge architectures to ensure the "deployability" claim holds across the fragmented IIoT hardware landscape.

Summary updated

Abstract

Enabled by the increasing availability of sensor data monitored from production machinery, condition monitoring and predictive maintenance methods are key pillars for an efficient and robust manufacturing production cycle in the Industrial Internet of Things. The employment of machine learning models to detect and predict deteriorating behavior by analyzing a variety of data collected across several industrial environments shows promising results in recent works, yet also often requires transferring the sensor data to centralized servers located in the cloud. Moreover, although collaborating and sharing knowledge between industry sites yields large benefits, especially in the area of condition monitoring, it is often prohibited due to data privacy issues. To tackle this situation, we propose an Autoencoder-based Federated Learning method utilizing vibration sensor data from rotating machines, that allows for a distributed training on edge devices, located on-premise and close to the monitored machines. Preserving data privacy and at the same time exonerating possibly unreliable network connections of remote sites, our approach enables knowledge transfer across organizational boundaries, without sharing the monitored data. We conducted an evaluation utilizing two real-world datasets as well as multiple testbeds and the results indicate that our method enables a competitive performance compared to previous results, while significantly reducing the resource and network utilization.
BibTeX
@inproceedings{becker2022federated,
  author = {Soeren Becker and Kevin Styp-Rekowski and Oliver Vincent Leon Stoll and Odej Kao},
  title = {{Federated Learning for Autoencoder-based Condition Monitoring in the Industrial Internet of Things}},
  booktitle = {IEEE Big Data},
  year = {2022},
  url = {https://openreview.net/forum?id=zUwOWoVIWA},
}

Loading related content...

Figure