An IoT Endpoint System-on-Chip for Secure and Energy-Efficient Near-Sensor Analytics

Francesco Conti, Robert Schilling, Pasquale Davide Schiavone, Antonio Pullini, Davide Rossi, Frank Kagan Gürkaynak, Michael Muehlberghuber, Michael Gautschi, Igor Loi, Germain Haugou, Stefan Mangard, Luca Benini

TL;DR

The "Fulmine" System-on-Chip (SoC) tackles the twin challenges of modern IoT endpoints: running heavy machine learning analytics locally while keeping the data and models strictly encrypted. By tightly coupling four programmable CPU cores with dedicated hardware accelerators for cryptography and Convolutional Neural Networks (CNNs) via shared memory, Fulmine achieves state-of-the-art energy efficiency (under 20 mW) without sacrificing flexibility or security.

What Problem Does This Paper Address?

IoT devices collect massive amounts of rich data (video, audio, biosignals). Sending all this raw data to the cloud for processing drains tiny batteries and clogs network bandwidth. The logical solution is "near-sensor analytics"—processing the data directly on the edge device to extract just the useful insights (e.g., sending a "face detected" tag instead of a 1080p video stream).

However, this creates a massive security vulnerability. The "distilled" data is highly privacy-sensitive, and the machine learning models (like CNN weights) stored on the device are valuable intellectual property. Protecting this data requires strong encryption. The problem is that running deep learning and cryptographic algorithms simultaneously requires billions of operations per second. Standard microcontrollers cannot handle this workload, and existing hardware accelerators are either too power-hungry or lack the flexibility to do both tasks securely within a tiny IoT power budget (typically under 20-30 mW).

Methods

The authors designed and fabricated a 65nm heterogeneous multi-core SoC named Fulmine. Its architecture relies on a few key innovations:

  • Tightly-Coupled Shared Memory: Instead of forcing the CPU cores and hardware accelerators to pass data back and forth over a slow bus, Fulmine uses a Tightly-Coupled Data Memory (TCDM). Four OpenRISC cores and two hardware accelerators share the same Level 1 memory, enabling "zero-copy" data exchange.
  • Hardware Cryptography Engine (HWCRYPT): A dedicated block supporting AES-128 (including the XTS mode used for secure memory storage) and Keccak (the basis for SHA-3).
  • Hardware Convolution Engine (HWCE): A specialized block for CNNs. Crucially, it is precision-scalable. While it defaults to 16-bit weights, it can be dynamically reconfigured to use 8-bit or 4-bit weights, allowing it to compute two or four convolutions simultaneously to save power and memory bandwidth.
  • Aggressive Power Management: The chip is divided into independent voltage and frequency domains. It utilizes granular clock-gating (shutting off power to specific sub-modules when not in use) and a deep-sleep mode for extreme duty-cycling.

Datasets

The paper evaluates the hardware using three real-world application workloads rather than training a new model on a novel dataset:

  1. Secure Autonomous Aerial Surveillance: Uses a ResNet-20 CNN on 224x224 images (similar to CIFAR-10 classification tasks).
  2. Local Face Detection: Uses 12-net and 24-net CNNs to detect faces before encrypting and sending the crops to a remote server.
  3. Seizure Detection: Uses multi-channel EEG biosignals processed via Principal Component Analysis (PCA), Wavelet Transforms, and a Support Vector Machine (SVM).

Key Insights and Ideas

  • Zero-copy is a game-changer for the edge: In low-power systems, moving data around often costs more energy than actually doing the math. By allowing the crypto engine, the CNN engine, and the CPUs to read/write to the exact same memory banks, Fulmine eliminates massive data-transfer overheads.
  • Software parallelization fails on secure memory encryption: The AES-XTS encryption mode is vital for protecting external memory (like CNN weights stored in Flash). However, unlike basic AES, XTS has internal data dependencies that make it nearly impossible to parallelize efficiently in software. Dedicated hardware for XTS is mandatory to prevent encryption from bottlenecking the entire ML pipeline.
  • Variable precision unlocks hardware reuse: CNNs are famously robust to lower precision. By designing the HWCE to dynamically switch to 4-bit weights, the system packs up to four times the operations into the same clock cycle without needing a completely separate, specialized binary-neural-network chip.

Key Results

  • Overall Efficiency: The chip consumes less than 20 mW on average at 0.8V. On a mixed workload of CNNs, encryption, and software filters, it achieves an incredible 5.74 pJ per equivalent RISC operation.
  • Comparison to State-of-the-Art: Fulmine beats the nearest competitor (SleepWalker) by executing the same workload 89x faster while maintaining superior energy efficiency.
  • UAV Surveillance Use Case: Running a secure ResNet-20 (where all weights and partial results are encrypted/decrypted on the fly) takes just 27 mJ per inference. A tiny nano-drone could run this continuously for a 7-minute flight using less than 0.25% of its total battery.
  • Medical IoT Use Case: The EEG seizure detection pipeline requires only 0.18 mJ per inference. A standard pacemaker battery could power this continuous monitoring for over 2 years.

Limitations and Future Work

  • Memory Bandwidth Bottlenecks: The authors note that in the ultra-efficient 4-bit CNN mode, the hardware accelerator completely maxes out the 4-port memory bandwidth. Scaling performance any further would require a wider, more complex memory interface.
  • The External Memory Problem: Even with a hyper-optimized SoC, the energy breakdown shows that accessing external Flash and FRAM memory still dominates the power budget for heavy workloads like ResNet-20.
  • Future Directions: To solve the external memory power drain, the authors suggest moving toward System-on-Package (SoP) designs—stacking the sensor, memory, and the Fulmine SoC in a single physical package to drastically reduce the energy cost of die-to-die connections.

Summary updated

Abstract

Near-sensor data analytics is a promising direction for internet-of-things endpoints, as it minimizes energy spent on communication and reduces network load - but it also poses security concerns, as valuable data are stored or sent over the network at various stages of the analytics pipeline. Using encryption to protect sensitive data at the boundary of the on-chip analytics engine is a way to address data security issues. To cope with the combined workload of analytics and encryption in a tight power envelope, we propose Fulmine, a system-on-chip (SoC) based on a tightly-coupled multi-core cluster augmented with specialized blocks for compute-intensive data processing and encryption functions, supporting software programmability for regular computing tasks. The Fulmine SoC, fabricated in 65-nm technology, consumes less than 20mW on average at 0.8V achieving an efficiency of up to 70pJ/B in encryption, 50pJ/px in convolution, or up to 25MIPS/mW in software. As a strong argument for real-life flexible application of our platform, we show experimental results for three secure analytics use cases: secure autonomous aerial surveillance with a state-of-the-art deep convolutional neural network (CNN) consuming 3.16pJ per equivalent reduced instruction set computer operation, local CNN-based face detection with secured remote recognition in 5.74pJ/op, and seizure detection with encrypted data collection from electroencephalogram within 12.7pJ/op.
BibTeX
@article{conti2017iot,
  author = {Francesco Conti and Robert Schilling and Pasquale Davide Schiavone and Antonio Pullini and Davide Rossi and Frank Kagan Gürkaynak and Michael Muehlberghuber and Michael Gautschi and Igor Loi and Germain Haugou and Stefan Mangard and Luca Benini},
  title = {{An IoT Endpoint System-on-Chip for Secure and Energy-Efficient Near-Sensor Analytics}},
  journal = {IEEE Trans. Circuits Syst. I Regul. Pap.},
  year = {2017},
  url = {https://openreview.net/forum?id=RGNRZp5CC__},
}

Loading related content...

Figure