Modern Large Language Model (LLM) inference is hitting a physical limit known as the "memory wall." Even the most advanced GPUs spend the vast majority of their energy budget simply moving data—specifically weight parameters—from memory to compute units. This paper introduces the Hardwired-Neurons Language Processing Unit (HNLPU), a paradigm shift that physically encodes LLM weights into the chip's metal interconnects, effectively turning the wiring itself into the model's memory.
The Challenge of Hardwired Intelligence
Historically, hardwiring a specific model into silicon was considered impractical. A 120-billion parameter model is too large for standard chip layouts, and the cost of unique photomasks for every model version could exceed $6 billion. However, as LLMs evolve into "general-purpose cognitive substrates," the authors argue that the trade-off—losing software flexibility for massive hardware efficiency—is finally justified.
Architectural Innovations: Metal-Embedding and Sea-of-Neurons
To overcome the barriers of scale and cost, the HNLPU utilizes two core innovations:
- Metal-Embedding (ME): Instead of using bulky 2D silicon transistors for storage, weights are embedded in the 3D topology of the chip's upper metal layers. This "Weights as Wires" approach increases density by 15×.
- Sea-of-Neurons Architecture: By using a structured ASIC approach, the researchers pre-fabricate a generic array of neurons using expensive high-resolution masks. The specific weights are only defined in the final, cheaper metal layers, allowing multiple models to share the bulk of the manufacturing costs.
Furthermore, the design employs bit-serialization and the distributive law to optimize arithmetic operations. By combining repeated multipliers, the silicon area required for computation is significantly reduced.
Performance and Sustainability
The HNLPU, implemented at 5nm, was evaluated against the NVIDIA H100 and the Cerebras WSE-3 using a 120B parameter Mixture-of-Experts (MoE) model.
The results represent a generational leap in performance:
- Energy Efficiency: Achieving 36 tokens/J, the HNLPU is 1,047× more efficient than the H100.
- Throughput: It delivers 249,960 tokens/s, a 5,555× boost over traditional GPU inference.
- Economic Impact: For large-scale deployments, the Total Cost of Ownership (TCO) is reduced by up to 80×, while the carbon footprint is slashed by 357×.
Limitations and the Future of "Mortal Computing"
The primary drawback of the HNLPU is weight immutability. Because the weights are physically part of the hardware, updating a model requires a physical "re-spin" of the metal layers, a process taking several weeks. The authors propose a "blue-green" deployment strategy to manage updates and suggest future iterations could include a small percentage of field-programmable neurons for "hotfixes."
By embracing "mortal computing"—where hardware and software are inseparable—the HNLPU offers a path to sustainable, high-performance AI that moves beyond the constraints of traditional von Neumann architectures.
Summary updated