Hardwired-Neurons Language Processing Units as General-Purpose Cognitive Substrates

Yang Liu, Yi Chen, Yongwei Zhao, Yifan Hao, Zifu Zheng, Weihao Kong, Zhangmai Li, Dongchen Jiang, Ruiyang Xia, Zhihong Ma, Zisheng Liu, Zhaoyong Wan, Yunqi Lu, Ximing Liu, Hongrui Guo, Zhihao Yang, Zhe Wang, Tianrui Ma, Mo Zou, Rui Zhang, Ling Li, Xing Hu, Zidong Du, Zhiwei Xu, Qi Guo, Tianshi Chen, Yunji ChenarXiv 2025 · DOI · arXiv

Modern Large Language Model (LLM) inference is hitting a physical limit known as the "memory wall." Even the most advanced GPUs spend the vast majority of their energy budget simply moving data—specifically weight parameters—from memory to compute units. This paper introduces the Hardwired-Neurons Language Processing Unit (HNLPU), a paradigm shift that physically encodes LLM weights into the chip's metal interconnects, effectively turning the wiring itself into the model's memory.

The Challenge of Hardwired Intelligence

Historically, hardwiring a specific model into silicon was considered impractical. A 120-billion parameter model is too large for standard chip layouts, and the cost of unique photomasks for every model version could exceed $6 billion. However, as LLMs evolve into "general-purpose cognitive substrates," the authors argue that the trade-off—losing software flexibility for massive hardware efficiency—is finally justified.

Architectural Innovations: Metal-Embedding and Sea-of-Neurons

To overcome the barriers of scale and cost, the HNLPU utilizes two core innovations:

  1. Metal-Embedding (ME): Instead of using bulky 2D silicon transistors for storage, weights are embedded in the 3D topology of the chip's upper metal layers. This "Weights as Wires" approach increases density by 15×.
  2. Sea-of-Neurons Architecture: By using a structured ASIC approach, the researchers pre-fabricate a generic array of neurons using expensive high-resolution masks. The specific weights are only defined in the final, cheaper metal layers, allowing multiple models to share the bulk of the manufacturing costs.
Arithmetic optimization using the distributive law to reduce silicon area.

Furthermore, the design employs bit-serialization and the distributive law to optimize arithmetic operations. By combining repeated multipliers, the silicon area required for computation is significantly reduced.

Performance and Sustainability

The HNLPU, implemented at 5nm, was evaluated against the NVIDIA H100 and the Cerebras WSE-3 using a 120B parameter Mixture-of-Experts (MoE) model.

Comparison of energy efficiency between traditional GPU clusters and the HNLPU.

The results represent a generational leap in performance:

  • Energy Efficiency: Achieving 36 tokens/J, the HNLPU is 1,047× more efficient than the H100.
  • Throughput: It delivers 249,960 tokens/s, a 5,555× boost over traditional GPU inference.
  • Economic Impact: For large-scale deployments, the Total Cost of Ownership (TCO) is reduced by up to 80×, while the carbon footprint is slashed by 357×.

Limitations and the Future of "Mortal Computing"

The primary drawback of the HNLPU is weight immutability. Because the weights are physically part of the hardware, updating a model requires a physical "re-spin" of the metal layers, a process taking several weeks. The authors propose a "blue-green" deployment strategy to manage updates and suggest future iterations could include a small percentage of field-programmable neurons for "hotfixes."

By embracing "mortal computing"—where hardware and software are inseparable—the HNLPU offers a path to sustainable, high-performance AI that moves beyond the constraints of traditional von Neumann architectures.

Summary updated

Abstract

The rapid advancement of Large Language Models (LLMs) has established language as a core general-purpose cognitive substrate, driving the demand for specialized Language Processing Units (LPUs) tailored for LLM inference. To overcome the growing energy consumption of LLM inference systems, this paper proposes a Hardwired-Neurons Language Processing Unit (HNLPU), which physically hardwires LLM weight parameters into the computational fabric, achieving several orders of magnitude computational efficiency improvement by extreme specialization. However, a significant challenge still lies in the scale of modern LLMs. A straightforward hardwiring of gpt-oss 120 B would require fabricating photomask sets valued at over 6 billion dollars, rendering this straightforward solution economically impractical. Addressing this challenge, we propose the novel Metal-Embedding methodology. Instead of embedding weights in a 2D grid of silicon device cells, Metal-Embedding embeds weight parameters into the 3D topology of metal wires. This brings two benefits: (1) a 15x increase in density, and (2) 60 out of 70 photomask layers are homogeneous across chips, including all EUV photomasks. In total, Metal-Embedding reduced the photomask cost by 112x, bringing the Non-Recurring Engineering (NRE) cost of HNLPU into an economically viable range. Experimental results show that HNLPU achieved 249,960 tokens/s (5,555x/85x that of GPU/WSE), 36 tokens/J (1,047x/283x that of GPU/WSE), 13,232 mm2 total die area, $59.46 M-123.5 M estimated NRE at 5 nm technology. Analysis shows that HNLPU achieved 41.7-80.4x improvement in cost-effectiveness and 357x reduction in carbon footprint compared to OpenAI-scale H100 clusters, under an annual weight updating assumption.
BibTeX
@misc{liu2025hardwired,
  author = {Yang Liu and Yi Chen and Yongwei Zhao and Yifan Hao and Zifu Zheng and Weihao Kong and Zhangmai Li and Dongchen Jiang and Ruiyang Xia and Zhihong Ma and Zisheng Liu and Zhaoyong Wan and Yunqi Lu and Ximing Liu and Hongrui Guo and Zhihao Yang and Zhe Wang and Tianrui Ma and Mo Zou and Rui Zhang and Ling Li and Xing Hu and Zidong Du and Zhiwei Xu and Qi Guo and Tianshi Chen and Yunji Chen},
  title = {{Hardwired-Neurons Language Processing Units as General-Purpose Cognitive Substrates}},
  year = {2025},
  eprint = {2508.16151},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.16151},
}
Figure