Abstract
The hardware-efficiency and accuracy of Deep Neural Networks (DNNs) implemented on In-memory Computing (IMC) architectures primarily depend on the DNN architecture and the peripheral circuit parameters. It is therefore essential to holistically co-search the network and peripheral parameters to achieve optimal performance. To this end, we propose XPert, which co-searches network architecture in tandem with peripheral parameters such as the type and precision of analog-to-digital converters, crossbar column sharing and the layer-specific input precision using an optimization-based design space exploration. Compared to VGG16 baselines, XPert achieves 10.24x (4.7x) lower EDAP, 1.72x (1.62x) higher TOPS/W, 1.93x (3x) higher TOPS/mm2 at 92.46% (56.7%) accuracy for CIFAR10 (TinyImagenet) datasets.
Introduction
In-Memory Computing (IMC) using resistive crossbars, such as RRAM, has emerged as a leading architectural paradigm to overcome the "memory wall"—the bottleneck caused by data movement between separate processing and memory units. By performing computations directly within the memory array, IMC offers the potential for massive parallelism and high energy efficiency. However, a significant challenge remains: while the crossbars themselves are highly efficient, the peripheral circuits—specifically Analog-to-Digital Converters (ADCs) and Digital-to-Analog Converters (DACs)—now account for the vast majority (often over 80%) of the system's total energy and area overhead.
Traditional design methodologies typically optimize the neural network architecture and the hardware hardware in isolation. Even contemporary "co-design" efforts often focus on a narrow subset of parameters, such as quantization levels, while ignoring critical hardware factors like ADC circuit types or physical area constraints. This paper introduces XPert, a framework designed for the holistic co-search of both software and hardware parameters to maximize efficiency without sacrificing accuracy.
The XPert Framework
XPert treats the hardware and software as a single, unified design space. To navigate this complex landscape, the framework utilizes a dual-phase optimization strategy based on a "SuperNet" approach, allowing for efficient exploration using gradient-based methods.
Phase 1: Structural Co-search
The first phase focuses on the physical layout and macro-architecture of the system. XPert searches for the optimal configuration of:
- Channel Depth (CD): The number of filters in each layer.
- Column Sharing (CS): The degree to which multiple crossbar columns share a single peripheral circuit.
- ADC Type (AT): Choosing between different circuit architectures, such as fast but area-intensive Flash ADCs or smaller but slower Successive Approximation Register (SAR) ADCs.
This phase is driven by a primary objective: minimizing delay under a strict, user-defined area constraint. By using XPertSim-Pytorch—a custom tool providing hardware-accurate, differentiable area and delay functions—the framework can use standard stochastic gradient descent (SGD) to find the optimal structural balance.
Phase 2: Precision Co-search
Once the physical structure is established, Phase 2 optimizes the data representation within that structure. It searches for layer-specific:
- ADC Precision (AP): The bit-resolution of the analog-to-digital conversion.
- Input Precision (IP): The bit-width of the activations passed between layers.
This phase prioritizes maintaining high classification accuracy while further pruning energy consumption and delay. Crucially, this phase is noise-aware, incorporating device variation and quantization noise into the search loop to ensure the resulting models are robust to the non-idealities inherent in analog hardware.
Experimental Results
The framework was evaluated using the CIFAR10 and TinyImagenet benchmarks, comparing XPert-optimized configurations against standard VGG16 baselines implemented with homogeneous (uniform) hardware settings.
Efficiency and Throughput
XPert demonstrated significant improvements across all primary hardware metrics:
- EDAP (Energy-Delay-Area Product): Achieved a 10.24× reduction for CIFAR10 and a 4.7× reduction for TinyImagenet.
- Energy Efficiency: Delivered up to 1.72× higher TOPS/W.
- Area Efficiency: Delivered up to 1.93× higher TOPS/mm².
Accuracy in Noisy Environments
One of the most striking findings was the improvement in accuracy when deployed on non-ideal hardware. By tailoring the precision and architecture to the specific needs of each layer, XPert improved accuracy by 9.7% on CIFAR10 (reaching 92.46%) and 4.7% on TinyImagenet (reaching 56.7%) compared to the baseline under the same noise conditions.
Key Insights
- The "Peripheral Tax" is Layer-Specific: XPert reveals that shallow layers require higher ADC and input precision to preserve raw feature information. In contrast, deeper layers can often tolerate lower precision and higher column sharing, allowing for significant area and energy savings.
- ADC Diversity: The optimal hardware configuration is rarely uniform. XPert frequently utilizes a mix of Flash and SAR ADCs, deploying Flash ADCs where speed is the bottleneck and SAR ADCs where area conservation is paramount.
- Area-Driven Architecture: The "optimal" neural network architecture is not fixed; it is a function of the available physical space. As area constraints tighten, XPert automatically adapts by reducing channel depths and increasing hardware sharing.
Conclusion
XPert demonstrates that the path to truly efficient In-Memory Computing lies in the simultaneous optimization of neural architectures and peripheral circuits. By breaking the silos between hardware and software design, XPert achieves order-of-magnitude improvements in efficiency while enhancing robustness to analog noise. Future work will look to extend this co-search methodology to more diverse architectures, such as ResNets and Transformers, and explore its applicability to different emerging memory technologies like PCM or STT-MRAM.
Summary updated