Precision-Energy-Throughput Scaling Of Generic Matrix Multiplication and Convolution Kernels Via Linear Projections

Mohammad Ashraful Anam, Paul N. Whatmough, Yiannis Andreopoulos

Abstract

Generic matrix multiplication (GEMM) and one-dimensional convolution/cross-correlation (CONV) kernels often constitute the bulk of the compute- and memory-intensive processing within image/audio recognition and matching systems. We propose a novel method to scale the energy and processing throughput of GEMM and CONV kernels for such error-tolerant multimedia applications by adjusting the precision of computation. Our technique employs linear projections to the input matrix or signal data during the top-level GEMM and CONV blocking and reordering. The GEMM and CONV kernel processing then uses the projected inputs and the results are accumulated to form the final outputs. Throughput and energy scaling takes place by changing the number of projections computed by each kernel, which in turn produces approximate results, i.e. changes the precision of the performed computation. Results derived from a voltage- and frequency-scaled ARM Cortex A15 processor running face recognition and music matching algorithms demonstrate that the proposed approach allows for 280%~440% increase of processing throughput and 75%~80% decrease of energy consumption against optimized GEMM and CONV kernels without any impact in the obtained recognition or matching accuracy. Even higher gains can be obtained if one is willing to tolerate some reduction in the accuracy of the recognition and matching applications.

Introduction

Modern multimedia applications—such as face recognition, music matching, and video processing—are increasingly demanding in terms of compute power and battery life. At their core, these applications rely on General Matrix Multiplication (GEMM) and Convolution (CONV) kernels. Traditionally, these kernels are implemented with full precision (e.g., 32-bit floating point). However, many of these applications are inherently error-tolerant (ET). Because input data is often noisy and the final output is a high-level decision, bit-perfect accuracy at every intermediate step is often unnecessary. This paper presents a systematic way to trade off computational precision for massive gains in speed and energy efficiency by scaling "Precision–Energy–Throughput" (PET) on off-the-shelf processors.

Methodology: Linear Projection-Based Scaling

The core innovation is the use of linear projections integrated directly into the data-handling stages of optimized kernels. This approach introduces a new layer (L2.5) in the software stack, positioned between top-level algorithm management and low-level optimized kernels.

System hierarchy showing the proposed L2.5 projection-based processing layer for PET scaling.

Projection-Based Compaction

Before performing the heavy lifting of multiplication or convolution, the input data is multiplied by a projection matrix (such as DCT-II or Haar). This concentrates the "energy" or most important information into a few coefficients. By performing energy compaction at the low-level kernel stage, the system essentially performs algorithmic "garbage collection" early, before expensive computations occur.

Kernel Execution and Accumulation

The GEMM or CONV operation is performed on these projected, smaller data blocks. The authors designed this to intercept the standard "blocking" and "reordering" phases used by high-performance libraries (like ATLAS or MKL).

Illustration of block partitioning in matrix multiplication (GEMM).

The results are then accumulated to form the final output. This integration allows the method to exploit SIMD (Single Instruction Multiple Data) instructions on standard CPUs, such as the ARM Neon architecture, with minimal overhead.

PET Scaling

The system can scale performance by simply changing the number of projections ($L$) used. Using fewer projections reduces the number of Multiply-Accumulate (MAC) operations and data transfers, saving energy and time at the cost of lower Signal-to-Noise Ratio (SNR). This allows for a smooth, "graceful" trade-off that can be tuned on-the-fly based on battery levels or real-time requirements.

Experimental Evaluation

The approach was evaluated on an ARM Cortex A15 processor using face recognition (Yale-A and Yale-B databases) and music matching (cover song datasets) tasks.

Key Results

Testing yielded dramatic improvements over standard optimized kernels:

  • Throughput: Achieved a 280% to 440% increase in processing speed.
  • Energy Efficiency: Reduced energy consumption by 75% to 80%.
  • Accuracy: In face recognition tests using 2D-PCA, the system achieved the exact same recognition rate as the lossless version while using only 1/8th of the data projections.
  • Signal Quality: For many tasks, using 6 out of 8 possible projections resulted in an SNR of ~70dB, effectively indistinguishable from lossless processing for multimedia applications.

Conclusion and Future Work

This methodology provides a general, software-adjustable way to scale performance on standard mobile processors. Future work includes exploring "data-driven" projections learned offline for specific datasets and implementing automated feedback mechanisms to adjust the number of projections in real-time to meet specific quality-of-service targets.

Summary updated

Abstract

Generic matrix multiplication (GEMM) and one-dimensional convolution/cross-correlation (CONV) kernels often constitute the bulk of the compute- and memory-intensive processing within image/audio recognition and matching systems. We propose a novel method to scale the energy and processing throughput of GEMM and CONV kernels for such error-tolerant multimedia applications by adjusting the precision of computation. Our technique employs linear projections to the input matrix or signal data during the top-level GEMM and CONV blocking and reordering. The GEMM and CONV kernel processing then uses the projected inputs and the results are accumulated to form the final outputs. Throughput and energy scaling takes place by changing the number of projections computed by each kernel, which in turn produces approximate results, i.e. changes the precision of the performed computation. Results derived from a voltage- and frequency-scaled ARM Cortex A15 processor running face recognition and music matching algorithms demonstrate that the proposed approach allows for 280%~440% increase of processing throughput and 75%~80% decrease of energy consumption against optimized GEMM and CONV kernels without any impact in the obtained recognition or matching accuracy. Even higher gains can be obtained if one is willing to tolerate some reduction in the accuracy of the recognition and matching applications.
BibTeX
@article{anam2014precision,
  author = {Mohammad Ashraful Anam and Paul N. Whatmough and Yiannis Andreopoulos},
  title = {{Precision-Energy-Throughput Scaling Of Generic Matrix Multiplication and Convolution Kernels Via Linear Projections}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2014},
  doi = {10.1109/tcsvt.2014.2321071},
}
Figure