EURETILE D7.3 - Dynamic DAL benchmark coding, measurements on MPI version of DPSNN-STDP (distributed plastic spiking neural net) and improvements to other DAL codes

Pier Stanislao Paolucci, Iuliana Bacivarov, Devendra Rai, Lars Schor, Lothar Thiele, Hoeseok Yang, Elena Pastorelli, Roberto Ammendola, Andrea Biagioni, Ottorino Frezza, Francesca Lo Cicero, Alessandro Lonardo, Francesco Simula, Laura Tosoratto, Piero ViciniarXiv 2014 · OpenReview

EURETILE Project Logo

Abstract

The EURETILE project required the selection and coding of a set of dedicated benchmarks. The project is about the software and hardware architecture of future many-tile distributed fault-tolerant systems. We focus on dynamic workloads characterised by heavy numerical processing requirements. The ambition is to identify common techniques that could be applied to both the Embedded Systems and HPC domains. This document is the first public deliverable of Work Package 7: Challenging Tiled Applications.

Introduction

As computing architectures evolve toward "many-tile" systems—incorporating hundreds or thousands of cores on a single platform—traditional static programming models increasingly struggle with dynamic workloads. These are applications where the required degree of parallelism fluctuates based on real-time demands, such as video decoding at variable resolutions or robotic control systems processing unpredictable sensory inputs.

The EURETILE project addresses these challenges by focusing on two primary areas:

  1. Dynamic Application Mapping: Developing methods to specify applications so they can automatically scale parallelism according to available hardware resources.
  2. Large-Scale Brain Simulation: Creating "mini-app" benchmarks capable of representing the extreme complexity of human brain dynamics (10^11 neurons, 10^15 synapses) to identify hardware bottlenecks for future specialized processors.

Methodology and Frameworks

The research utilizes two core software frameworks to evaluate many-core performance:

Distributed Application Layer (DAL)

DAL employs a scenario-based design flow using Expandable Process Networks (EPNs). EPNs allow a single application specification to be "unfolded" into varying numbers of processes. This flexibility enables the system to optimize the degree of parallelism based on the current hardware state and resource availability.

DPSNN-STDP (Distributed Plastic Spiking Neural Net)

DPSNN-STDP is a natively distributed C++/MPI simulation of a Spiking Neural Network. It utilizes a hybrid simulation approach:

  • Time-driven simulation: Manages the continuous membrane potential of neurons using the computationally efficient Izhikevich model.
  • Event-driven simulation: Handles synaptic plasticity via Spike-Timing-Dependent Plasticity (STDP).

The simulation is structured as a 2D grid of "neural columns," each containing 1,000 neurons. This architecture facilitates predictable local communication while testing the system's capacity for massive, sparse data exchange.

Benchmark Datasets

The frameworks were tested using standardized inputs:

  • Ray Tracing: The classic Cornell Box scene.
  • H.264 Codec: Video sequences at QCIF and CIF resolutions.
  • Neural Networks: Synthetic networks ranging from 200,000 to 6.6 billion synapses (up to 32.8 million neurons), designed for consistent spiking behavior across varying core counts.

Key Results and Performance Analysis

Scaling and Efficiency

The DPSNN-STDP code demonstrated significant scalability. It simulated one second of activity for a network of 1.6 billion synapses in 115 wall-clock seconds using 128 hardware cores, achieving a 41.5x speedup compared to a single core. Profiling revealed that communication overhead (via MPI_Alltoall) accounted for less than 2% of execution time, indicating a highly optimized communication strategy.

In the DAL framework, Ray Tracing achieved a speedup of nearly 20x on 24 cores. The overhead introduced by the DAL abstraction layer was minimal, performing only ~8% slower than highly optimized OpenMP implementations.

The "Jitter" Bottleneck

A critical finding in the brain simulation was that performance deviations from ideal scaling were primarily caused by workload imbalance rather than network latency. This "jitter" occurs because biological neural activity is inherently unpredictable; some neurons fire more frequently, causing certain CPU cores to finish tasks early and remain idle at synchronization barriers.

Resource Congestion

Experiments with the H.264 encoder showed that increasing parallelism does not always yield proportional gains. Using 11 Motion Estimation (ME) processes provided nearly the same speedup as 5 processes, suggesting that resource congestion in the cache and memory bus becomes a limiting factor beyond a certain threshold.

Conclusion and Future Directions

The EURETILE project successfully demonstrated that Expandable Process Networks and optimized MPI-based simulations can effectively manage dynamic workloads on many-core architectures. However, several challenges remain:

  • Connectome Complexity: Future simulations must incorporate "white matter" connectomes—long-range connections that will complicate spike delivery.
  • Dynamic Load Balancing: To mitigate the "jitter" bottleneck, future research will focus on real-time migration of neural clusters between cores.
  • Hardware Co-design: These results inform the development of specialized hardware capable of better supporting sparse memory access and hardware-level synchronization.
Project Funding and Context

This work was supported by the European Commission under the Seventh Framework Programme (FP7).

Summary updated

Abstract

The EURETILE project required the selection and coding of a set of dedicated benchmarks. The project is about the software and hardware architecture of future many-tile distributed fault-tolerant systems. We focus on dynamic workloads characterised by heavy numerical processing requirements. The ambition is to identify common techniques that could be applied to both the Embedded Systems and HPC domains. This document is the first public deliverable of Work Package 7: Challenging Tiled Applications.
BibTeX
@misc{paolucci2014euretile,
  author = {Pier Stanislao Paolucci and Iuliana Bacivarov and Devendra Rai and Lars Schor and Lothar Thiele and Hoeseok Yang and Elena Pastorelli and Roberto Ammendola and Andrea Biagioni and Ottorino Frezza and Francesca Lo Cicero and Alessandro Lonardo and Francesco Simula and Laura Tosoratto and Piero Vicini},
  title = {{EURETILE D7.3 - Dynamic DAL benchmark coding, measurements on MPI version of DPSNN-STDP (distributed plastic spiking neural net) and improvements to other DAL codes}},
  year = {2014},
  eprint = {1408.4587},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1408.4587},
}
Figure