Precision-Energy-Throughput Scaling of Generic Matrix Multiplication and Convolution Kernels via Linear Projections

Mohammad Ashraful Anam, Paul N. Whatmough, Yiannis Andreopoulos

TL;DR

This paper introduces a novel method to dynamically scale the precision of matrix multiplication (GEMM) and convolution (CONV) kernels, crucial for multimedia processing. By applying linear projections to input data, they achieve significant improvements in processing throughput and energy efficiency with minimal impact on accuracy for error-tolerant applications.

What Problem Does This Paper Address?

Modern multimedia applications, especially in areas like image and audio recognition, rely heavily on computationally intensive kernels like GEMM and CONV. These systems are often "error-tolerant," meaning they can tolerate some approximation in their results without significantly degrading overall performance. However, existing methods for optimizing these kernels for energy and throughput often lack flexibility, are application-specific, or require specialized hardware. This paper addresses the challenge of achieving significant, yet dynamic, precision-energy-throughput (PET) scaling in generic GEMM and CONV kernels that can be easily deployed on commercial off-the-shelf processors.

Methods

The core innovation lies in applying linear projection techniques to the input data before it's processed by the GEMM or CONV kernels. This is integrated into the top-level processing stage, specifically during data partitioning and reordering.

  • Linear Projections: The authors introduce projection matrices (e.g., DCT coefficients) that transform the input data into a more compact representation. This effectively reduces the "effective precision" of the computation.
  • Hierarchical Representation: These projections create a hierarchical representation of the inner-product computations, which are fundamental to GEMM and CONV.
  • Software-Adjustable Precision: The number of projections used directly controls the trade-off between precision, energy consumption, and throughput. This allows for dynamic, software-based tuning of these parameters.
  • Integration with Existing Kernels: The approach is designed to be seamlessly integrated into existing optimized GEMM and CONV kernels, leveraging SIMD instructions for efficient execution without breaking the data blocking and reordering patterns.

Datasets

The paper evaluates its approach using:

  • Face Recognition: Yale-A and Yale-B face image databases were used for experiments with the 2D-PCA algorithm.
  • Music Matching: A music identification application that uses beat and tempo data for cross-correlation was used.

The authors do not introduce new benchmarks but rather evaluate their method on established applications and datasets.

Key Insights and Ideas

  • Dynamic Precision Scaling: The ability to dynamically adjust the precision of GEMM and CONV kernels through a simple parameter (number of projections) is a significant advancement. This allows for fine-grained control over the PET trade-off.
  • Energy Compaction as a Key Enabler: The paper highlights that many error-tolerant multimedia applications inherently perform energy compaction. The proposed projection method complements this, allowing for substantial performance gains without degrading the final application accuracy.
  • Generality of the Approach: The method is not tied to a specific algorithm but is applicable to any application that relies on GEMM and CONV kernels, making it broadly useful.
  • Minimal Impact on Existing Workflows: The integration with existing SIMD-optimized kernels means that the benefits can be realized on common embedded processors without requiring specialized hardware.

Key Results

The proposed approach demonstrates significant improvements:

  • GEMM:
    • Up to 533% increase in processing throughput and 85% reduction in energy consumption compared to conventional GEMM kernels.
  • Achieved with minimal impact on recognition accuracy (SNR of ~70dB for 6 projections).
  • Even higher gains (up to 850% throughput increase and 91% energy reduction) were observed with more projections, though this led to a slight decrease in accuracy.
  • CONV:
    • Up to 440% increase in processing throughput and 82% reduction in energy consumption for audio matching applications.
  • These results were achieved with a matching accuracy comparable to conventional methods.
  • Application-Level Benefits: In face recognition, the approach led to a 350% increase in throughput and a 79% decrease in overall energy consumption for the entire application.

Limitations and Future Work

  • Accuracy Degradation with More Projections: While the approach offers substantial gains, using a very small number of projections (e.g., L=12 or L=16) can lead to a noticeable decrease in recognition accuracy, which might be undesirable for some applications. The authors suggest L=8 as a good operational point.
  • Non-Adaptive Transforms: The current work uses non-adaptive transforms (like DCT and Haar). Future work could explore using adaptive transforms derived from principal component analysis (PCA) based on specific training datasets for even more optimized scaling.
  • Learning Projection Parameters: The authors suggest that projection parameters could be tuned to learn the best settings based on feedback about incurred imprecision from the application, enabling a more sophisticated dynamic adaptation.

Summary updated

Abstract

Generic matrix multiplication (GEMM) and convolution (CONV)/cross-correlation kernels often constitute the bulk of the compute- and memory-intensive processing within image/audio recognition and matching systems. We propose a novel method to scale the energy and processing throughput of GEMM and CONV kernels for such error-tolerant multimedia applications by adjusting the precision of computation. Our technique employs linear projections to the input matrix or signal data during the top-level GEMM and CONV blocking and reordering. The GEMM and CONV kernel processing then uses the projected inputs and the results are accumulated to form the final outputs. Throughput and energy scaling takes place by changing the number of projections computed by each kernel, which in turn produces approximate results, i.e., changes the precision of the performed computation. Results derived from a voltage- and frequency-scaled ARM Cortex A15 processor running face recognition and music-matching algorithms demonstrate that the proposed approach allows for a 280%–440% increase of processing throughput and a 75%–80% decrease of energy consumption against the optimized GEMM and CONV kernels without any impact on the obtained recognition or matching accuracy. Even higher gains can be obtained, if one is willing to tolerate some reduction in the accuracy of the recognition and matching applications.
BibTeX
@article{anam2014precision,
  author = {Mohammad Ashraful Anam and Paul N. Whatmough and Yiannis Andreopoulos},
  title = {{Precision-Energy-Throughput Scaling of Generic Matrix Multiplication and Convolution Kernels via Linear Projections}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2014},
  doi = {10.1109/tcsvt.2014.2321071},
}
Figure