TL;DR
This paper introduces GENIAL, a novel machine learning framework that uses network inversion to automatically discover new binary encodings for arithmetic logic units (ALUs), specifically multipliers. By efficiently exploring the design space, GENIAL achieves significant power savings (up to 18%) and area improvements compared to traditional encodings like two's complement, paving the way for more energy-efficient AI hardware.
What Problem Does This Paper Address?
The ever-increasing demand for AI workloads, especially from large language models (LLMs), puts a massive strain on computational resources and energy consumption. Optimizing the core arithmetic units within these systems is crucial for reducing their power footprint. Traditional hardware design methods struggle to explore the vast space of possible binary encodings for representing numbers, often relying on well-established but not necessarily power-optimal formats like two's complement. This paper tackles the challenge of automatically discovering novel, power-efficient operand encodings that can significantly reduce switching activity, a major contributor to power consumption in digital circuits, particularly for AI workloads where data distributions are often centered around zero.
Methods
GENIAL employs a multi-stage approach centered around a Transformer-based surrogate model. This model is trained in two phases: first, self-supervised pretraining helps it learn robust representations of hardware designs from their abstracted encodings. Second, supervised finetuning allows it to accurately predict key hardware metrics like power and area. The core innovation lies in "network inversion": by treating the trained surrogate model as a black box, GENIAL uses gradient-based optimization to "invert" it. This process iteratively adjusts input encodings to minimize the predicted power consumption, effectively searching for optimal designs.
To ensure the generated encodings are valid and practical, GENIAL incorporates:
- Legality-aware losses: These regularization terms encourage the generated encodings to map values to distinct binary representations, preventing ambiguity.
- Hierarchical EDA flows: GENIAL integrates multiple Electronic Design Automation (EDA) flows with varying levels of fidelity and runtime. A fast "ESPRS" flow screens many candidates, a mid-fidelity "OPENR" flow evaluates promising ones for area and power, and a high-effort "FLOWY" flow performs detailed logic synthesis optimization (LSO) for the best candidates, focusing on switching activity (SwAct) and power.
- Data Augmentation: Techniques like swapping input bit positions and inverting encodings are used to create more training data, improving the surrogate model's accuracy.
Datasets
The paper doesn't explicitly mention large, pre-existing public datasets for training hardware encodings. Instead, GENIAL builds its dataset iteratively.
- Initial Dataset: A set of random, valid designs are generated and their Quality of Results (QoR) are extracted to form the initial training data.
- Augmented Datasets: During the iterative process, newly evaluated designs are added to the dataset, expanding its diversity and improving the surrogate model's accuracy over time. For experiments, they used datasets of 10k randomly generated designs, augmented with additional designs from previous GENIAL runs.
Key Insights and Ideas
- Network Inversion for Hardware Design: Applying network inversion, a technique more common in computer vision, to discover hardware encodings is a novel approach that significantly accelerates design space exploration.
- Operand Encoding as a Key Optimization Lever: The paper highlights that the choice of binary representation for numbers has a substantial impact on power consumption, often overlooked in favor of optimizing circuit architecture.
- Synergy of ML and EDA: GENIAL effectively bridges the gap between machine learning and traditional EDA tools. The ML model guides the search, while the EDA tools provide the ground truth for training and evaluation.
- Hierarchical EDA for Scalability: Using a tiered approach to EDA flows (fast screening to high-effort optimization) is crucial for making the computationally intensive design space exploration feasible.
- Switching Activity as a Powerful Proxy: The paper demonstrates that switching activity (SwAct) is a strong indicator of dynamic power consumption, allowing for efficient optimization even without full power simulations for every candidate.
Key Results
- Significant Power Savings: GENIAL discovered novel encodings that achieve up to 18% switching activity savings within multipliers compared to the conventional two's complement encoding.
- Competitive with State-of-the-Art: One discovered encoding closely matched the energy efficiency of the Sign-Magnitude Extended (SME) format, a known efficient encoding.
- Superior Sample Efficiency: GENIAL consistently outperformed random search, binary switching search, and a naive surrogate-based approach in terms of finding optimal designs with fewer processed samples.
- Generalizability: The framework showed promise by achieving improvements on Finite State Machines (FSMs), indicating its potential beyond multipliers.
- Pareto Front Discovery: For 4-bit multipliers, GENIAL reached the Pareto front for Switching Activity (SwAct) and area.
Limitations and Future Work
- Runtime of High-Fidelity Flows: The most accurate EDA flows (like FLOWY) are computationally expensive, limiting the scale of exploration. Future work could focus on parallelization, lightweight proxies, or adaptive flow selection.
- LUT-Based Representation: Candidate multipliers are generated as Look-Up Tables (LUTs), which might not be as optimal as hand-tuned RTL implementations before synthesis. Investigating direct RTL templates or hybrid representations is suggested.
- Benchmark Scale: Experiments were primarily conducted on 4-bit multipliers and randomly generated FSMs. Future work should validate the approach on larger and more diverse circuits (e.g., 8- or 16-bit multipliers, square-root units).
- Single-Objective Guidance: The current pipeline uses a single proxy metric for optimization. Incorporating multiple proxies or an uncertainty-aware exploration strategy could lead to discovering even better designs.
- Exploration Capability: While GENIAL found competitive encodings, it did not consistently surpass the SME format. Enhancing search diversity and considering alternative constraints could unlock superior designs.
The authors envision future work scaling GENIAL to larger circuits, exploring different input data distributions, and applying it to a wider range of logic functions beyond arithmetic units.
Summary updated