Computer architecture has long been defined by a strict trade-off: the tension between flexibility and efficiency. General-purpose central processing units (CPUs) can execute any computable algorithm because they rely on a highly malleable software instruction stream. However, this flexibility comes at a steep cost. For every useful arithmetic operation performed, a CPU expends significant energy fetching instructions, decoding them, managing loop counters, and calculating memory addresses.
Conversely, Application-Specific Integrated Circuits (ASICs) eliminate this overhead by hardwiring the underlying logic to a specific task. While ASICs achieve maximum energy efficiency, they are completely rigid. If a neural network architecture changes or a new wireless protocol emerges, the hardware becomes obsolete. As highlighted by Venieris et al., bridging this gap requires a cross-stack approach that infuses programmable architectures with the efficiency of custom hardware.
Stripping Away Control Overhead via ISA Extensions
To understand why modern custom processors can approach ASIC-like efficiency while remaining programmable, we must look at how they eliminate instruction overhead. In a standard processor, executing a matrix multiplication loop requires constant branch checks, data shuffling, and address increments.
Zaruba solves this in the "Snitch" architecture by decoupling memory orchestration from the execution unit. By introducing Stream Semantic Registers (SSRs) and Floating-point Repetition (FREP), the architecture allows a tiny integer core to offload memory addressing and loop management. Instead of issuing explicit load and loop instructions, the hardware automatically streams data from memory directly into the floating-point unit (FPU) registers. This "pseudo dual-issue" mechanism ensures the FPU remains fully utilized without burning energy on control logic.
This philosophy of targeted Instruction Set Architecture (ISA) extensions is widely applicable. For instance, Gautschi et al. extend a RISC-V core with custom DSP instructions, achieving a 3.2× energy efficiency boost for edge applications. Similarly, Rutishauser et al. introduce lightweight ISA extensions for Ternary Neural Networks, gaining a 57% energy efficiency improvement with a negligible 0.9% silicon area overhead. By baking only the most repetitive, computationally expensive patterns into hardware—such as data-shuffling or tensor products—architectures like Flex-V by Nadalini et al. and DARKSIDE by Garofalo preserve full programmability for evolving model architectures while shedding the traditional CPU power penalty.
Exposing the Datapath to the Compiler
Another way to bypass the efficiency penalty of general-purpose architectures is to change how software interacts with hardware. Traditional ISAs act as an abstraction layer that hides the physical routing of data between registers and functional units. This abstraction requires complex hardware control logic to prevent data hazards.
"Exposed datapath" architectures turn this paradigm on its head. As explained by Jääskeläinen et al., letting the compiler precisely control data movement eliminates the need for "black box" hardware control logic. Transport Triggered Architecture (TTA) is a prime example of this mechanism. In a TTA system, the only instruction is a conditional move (MOVE). Operations are triggered as a side effect of moving data into a functional unit's input register.
Multanen et al. demonstrate that by exposing the datapath and utilizing a software-managed Instruction Register File, TTA cores can reduce energy consumption by up to 64% compared to standard RISC-V cores. Because the compiler knows exactly when and where data is needed, it bypasses the traditional register file entirely, routing outputs directly to the inputs of the next operation. This spatial orchestration allows massively parallel arrays, such as the vision processor designed by Safarpour et al., to maintain software programmability while operating at the ultra-low energy thresholds typically reserved for fixed-function circuits.
Strategic and Formalized Flexibility
A common misconception is that a system must be infinitely flexible to be useful. In reality, hardware designers can achieve near-ASIC performance by identifying and supporting only a strategic subset of configurations.
Kao formalizes this concept by breaking down DNN accelerator flexibility into four mathematical axes: Tiling, Order, Parallelism, and Shape (TOPS). By quantifying these dimensions, the author demonstrates that hardware does not need to support arbitrary execution patterns. Instead, designing hardware to support a targeted envelope of tile sizes and array shapes captures the vast majority of an application's performance benefits while minimizing physical area overhead.
This principle of bounded versatility is critical when designing multi-standard systems. For example, Guo et al. show that a multi-standard QC-LDPC decoder can support diverse wireless protocols (like WiMAX and WiFi) without sacrificing gigabit-per-second throughput. Rather than building a fully generic processor, the authors implement a configurable multi-bank memory and a simplified permutation network. The system remains flexible enough to switch between protocols but rigid enough to maintain an optimized throughput-to-area ratio.
Heterogeneous Scaling and Co-Design
When scaling these principles to massive workloads like 5G software-defined radio or large-scale machine learning, architectures must combine flexible control clusters with specialized streaming fabrics.
To prevent memory bottlenecks from stalling execution, architectures like MemPool by Riedel et al. utilize low-latency shared L1 memory pools across hundreds of cores. To maximize floating-point utilization on irregular workloads, Paulin et al. couple RISC-V clusters with specialized hardware "Streaming Units" in the Occamy architecture, achieving 83% FPU utilization on sparse data and stencil computations.
Ultimately, the gap between programmability and efficiency is bridged by tightly coupling distinct computational paradigms. Whether it is combining analog neuro-synaptic cores with digital microprocessors to accelerate spiking neural networks (Billaudelle et al.), linking CPUs and FPGAs via cache-coherent links (Roscoe et al.), or standardizing streaming interfaces between modular robotic hardware nodes (Mayoral-Vilches et al.), modern architecture is moving away from monolithic design. By delegating control logic to lightweight programmable cores and data orchestration to specialized hardware primitives, these systems achieve software-defined flexibility at hardware-defined efficiency.
Key Foundational Work
Achieving Flexibility, Efficiency, and Generality in Blackboard Architectures (1987) — contributed 1 concept elements to this direction.
NON-VONs Applicability to Three AI Task Areas (1985) — contributed 1 concept elements to this direction.