TL;DR
This paper introduces "TimeFloats," a novel train-in-memory architecture that performs floating-point scalar products directly within memory using time-domain signal processing. This approach significantly boosts energy efficiency for on-device deep learning training, a critical step towards sustainable AI at the edge.
What Problem Does This Paper Address?
Deep learning models have become incredibly powerful but demand substantial computational resources for training, often requiring energy-intensive GPUs and leading to significant carbon emissions. While compute-in-memory (CIM) architectures have shown promise for efficient inference by performing computations within memory arrays, enabling training directly on edge devices remains a major challenge. Traditional CIM approaches often rely on analog-to-digital converters (ADCs) and digital-to-analog converters (DACs), which add complexity and power consumption, especially at advanced CMOS nodes. Furthermore, training neural networks, particularly backpropagation, requires floating-point arithmetic to handle gradients, which is more complex than the fixed-point operations typically supported by existing CIM designs. This paper tackles the need for efficient, on-device training of deep neural networks by proposing a CIM architecture that supports floating-point operations using time-domain signal processing, thereby avoiding complex ADCs/DACs and leveraging digital building blocks.
Methods
The core innovation of TimeFloats lies in its use of time-domain signal processing for floating-point scalar products, enabling both storage and computation within the same memory structures. The architecture breaks down the floating-point multiplication and accumulation (MAC) operation into five key steps:
- Element-wise Exponent Addition: Input and weight exponents are added using analog RC circuits, with the resulting sum represented by the discharge time of the circuit.
- Largest Exponent Detection: A digital circuit (using D Flip-Flops and multiplexers) identifies the largest summed exponent across all operations.
- Time-Domain Exponent Normalization and Mantissa Scaling: The largest exponent is used to normalize other exponents by subtracting it in the time domain. This normalization is then applied to the mantissa bits through right-shifting, effectively dividing by powers of two.
- Fixed-Point Input-Weight Mantissa Scalar Product: The scaled mantissas are then multiplied and accumulated using a passive memristor crossbar. This step leverages the physics of memristors to perform MAC operations by accumulating charge.
- Digitization and Reformatting: The accumulated charge (representing the product-sum) is digitized using a single shared ADC and reformatted into a floating-point number.
A key architectural advantage is the ability to vertically fold the design, placing the passive memristor crossbar at the back-end-of-line (BEOL) and the digital peripherals at the front-end-of-line (FEOL), leading to higher integration density.
Datasets
Not discussed in the paper. The paper focuses on the architecture and its simulation results, not on training specific models on particular datasets.
Key Insights and Ideas
- Time-Domain for Floating-Point CIM: The paper demonstrates that time-domain signal processing can be effectively used to implement complex floating-point operations (specifically scalar products) within a CIM architecture, bypassing the need for traditional ADCs/DACs.
- Digital Building Blocks with Analog Benefits: By using time-domain representations for analog quantities, TimeFloats primarily relies on digital building blocks, which aids in scalability and integration with existing digital circuits, while still benefiting from the low-power, physics-based computations of analog systems.
- Vertical Folding for Area Efficiency: The proposed architecture's ability to place the memristor crossbar at BEOL and peripherals at FEOL is a significant design insight for achieving high integration density on limited silicon footprints, crucial for edge devices.
- Process Variability Sensitivity: The analysis highlights that exponent computations are more sensitive to process variability than mantissa computations, guiding future optimization efforts towards mitigating exponent variability.
Key Results
The TimeFloats architecture achieves an energy efficiency of 22.1 TOPS/W when evaluated on a 15nm CMOS technology. This efficiency is attributed to the time-domain operations and the simplified architecture. The paper also provides a breakdown of energy consumption for different operational steps, showing that the mixed-signal exponent adder and the largest exponent detector are significant contributors to the overall energy budget.
Limitations and Future Work
The authors acknowledge that analog operations, even with time-domain representations, are susceptible to process variability (e.g., in transistors and memristors), which can degrade accuracy. They suggest that calibration mechanisms can mitigate this, but edge devices have inherent resource constraints.
Future work includes:
- Exploiting the exponent detector for predictive early termination in crossbar processing to further reduce power consumption.
- Customizing the design of the largest exponent detector to lower its power usage, as it is currently the largest power contributor.
- Further optimization for other floating-point precisions and exploring its application in training various deep neural network architectures.
Summary updated