OutlierNets: Highly Compact Deep Autoencoder Network Architectures for On-Device Acoustic Anomaly Detection

Saad Abbasi, Mahmoud Famouri, Mohammad Javad Shafiee, Alexander Wong

TL;DR

OutlierNets are a new family of ultra-compact convolutional autoencoders designed specifically for detecting industrial machinery failures via sound. By using machine-driven design exploration, these models achieve state-of-the-art accuracy with as few as 686 parameters—representing a 5800x reduction in complexity and up to 21x lower latency compared to previous benchmarks.

What Problem Does This Paper Address?

Acoustic Anomaly Detection (AAD) is a cornerstone of "Industry 4.0," allowing factories to monitor machine health by listening for irregular sounds. While deep learning models excel at this, they are typically "heavyweight." Standard architectures like ResNet-18 or even tailored 4-million-parameter autoencoders require significant memory and computational power.

This creates a deployment gap: these models often can't run on the resource-constrained edge devices (like microcontrollers or low-power CPUs) found on factory floors. Furthermore, for safety-critical applications, detection must happen in real-time; a laggy model might miss the window to shut down a failing machine safely.

Methods

The authors move away from manual architecture design, instead employing a machine-driven design exploration strategy called generative synthesis.

  1. Generative Synthesis: The researchers defined a set of operational constraints (e.g., parameter count under 100,000 and high detection accuracy) and used an automated generator to traverse the architectural search space.
  2. Macro & Micro-Architecture: The resulting OutlierNets utilize highly efficient building blocks, including depthwise convolutions, pointwise convolutions, and "replicators."
  3. Autoencoder Logic: Like standard AAD models, these are trained in an unsupervised manner on "normal" sounds. At inference, the model tries to reconstruct the input sound; if the reconstruction error is high, the sound is flagged as an anomaly.
  4. Input Processing: Audio is converted into Mel-spectrograms and cropped into one-second windows (32x128), ensuring the system only needs a brief snippet of audio to make a decision.

Datasets

The study evaluates OutlierNets using the MIMII dataset, a standard benchmark for industrial machine sounds.

  • Machine Types: The paper focuses specifically on Fans and Sliders.
  • Data Split: The training set contains only "normal" recordings, while the test set is a balanced mix of normal and malfunctioning sounds across various Signal-to-Noise Ratios (SNRs).

Key Insights and Ideas

  • Extreme Efficiency via Diversity: The machine-driven process didn't just make the models smaller; it introduced high "micro-architecture diversity." The models use different types of layers in unconventional sequences that a human designer might not consider, but which are mathematically optimal for the task.
  • Task-Specific Tailoring: The exploration found that different machines require different complexities. For example, "Slider" machines were a "tougher" task, leading the generator to produce more complex architectures (with dense latent spaces) compared to the "Fan" models.
  • The Power of Small Windows: By using one-second audio windows, the authors minimize "inherent lag," ensuring the system is viable for real-time safety shutdowns.

Key Results

The authors compared OutlierNets against a state-of-the-art Convolutional Autoencoder (CAE-MCS) which has 4 million parameters.

  • Complexity: The smallest OutlierNet has only 686 parameters and a model size of 2.7 KB. This is small enough to fit into the Static RAM (SRAM) of almost any modern microcontroller.
  • Accuracy: Despite being orders of magnitude smaller, OutlierNets achieved an average AUC (Area Under Curve) of 88.8% on sliders, matching or exceeding the 4M-parameter baseline.
  • Latency:
    • On an Intel Core i5, OutlierNets achieved a runtime of 0.366 $\mu$s (21x faster than the baseline).
  • On an ARM Cortex A-72 (an embedded CPU), the fastest OutlierNet clocked in at 7.2 $\mu$s, compared to 78.2 $\mu$s for the baseline.

Limitations and Future Work

While the results are impressive, the authors note a few areas for expansion:

  • Machine Variety: This specific study focused on fans and sliders. Future work will explore if these compact architectures generalize well to other industrial sounds like valves or pumps.
  • Ultra-Low Power: The researchers aim to push the boundaries even further, looking at design considerations for even lower-power microcontrollers to enable "always-on" acoustic monitoring with minimal battery drain.

Summary updated

Abstract

Human operators often diagnose industrial machinery via anomalous sounds. Given the new advances in the field of machine learning, automated acoustic anomaly detection can lead to reliable maintenance of machinery. However, deep learning-driven anomaly detection methods often require an extensive amount of computational resources prohibiting their deployment in factories. Here we explore a machine-driven design exploration strategy to create OutlierNets, a family of highly compact deep convolutional autoencoder network architectures featuring as few as 686 parameters, model sizes as small as 2.7 KB, and as low as 2.8 million FLOPs, with a detection accuracy matching or exceeding published architectures with as many as 4 million parameters. The architectures are deployed on an Intel Core i5 as well as a ARM Cortex A72 to assess performance on hardware that is likely to be used in industry. Experimental results on the model’s latency show that the OutlierNet architectures can achieve as much as 30× lower latency than published networks.
BibTeX
@inproceedings{abbasi2021outliernets,
  author = {Saad Abbasi and Mahmoud Famouri and Mohammad Javad Shafiee and Alexander Wong},
  title = {{OutlierNets: Highly Compact Deep Autoencoder Network Architectures for On-Device Acoustic Anomaly Detection}},
  booktitle = {Sensors},
  year = {2021},
  url = {https://openreview.net/forum?id=5rdAT93LrA},
}

Loading related content...

Figure