Erratum to: CMN: a co-designed neural architecture search for efficient computing-in-memory-based mixture-of-experts

Shihao Han, Sishuo Liu, Shucheng Du, Mingzi Li, Zijian Ye, Xiaoxin Xu, Yi Li, Zhongrui Wang, Dashan Shang

TL;DR

This erratum corrects a previous publication, highlighting that the article "CMN: a co-designed neural architecture search for efficient computing-in-memory-based mixture-of-experts" was inadvertently published without open access. The correction ensures the article is now freely available, allowing wider dissemination of its findings on designing efficient neural architectures for computing-in-memory systems.

What Problem Does This Paper Address?

The original article, now corrected and made open access, addresses the challenge of designing efficient neural network architectures specifically for computing-in-memory (CIM) hardware. CIM systems aim to perform computations directly where data is stored, reducing data movement and energy consumption. However, traditional neural network architectures are not always optimal for the unique constraints and characteristics of CIM hardware. This paper focuses on developing a co-designed approach, where the neural architecture search (NAS) process is integrated with the CIM hardware design to create architectures that are both performant and energy-efficient on these specialized platforms. The "mixture-of-experts" (MoE) paradigm is employed, which can offer improved efficiency and scalability by routing inputs to specialized sub-networks. The core problem is bridging the gap between general-purpose neural network design and the specific requirements of efficient CIM implementation.

Methods

The paper introduces a "co-designed neural architecture search" (CMN) approach. This method integrates the search for optimal neural network architectures with the design considerations of computing-in-memory hardware. While the specific details of the search algorithm and the co-design methodology are not elaborated upon in this erratum, it implies a process where the NAS is aware of and optimized for CIM constraints. The use of a "mixture-of-experts" (MoE) architecture is central, suggesting that the search aims to find efficient configurations of these expert sub-networks for CIM. This approach likely involves a search space that includes architectural choices relevant to CIM, such as layer types, connectivity, and parameters that can be efficiently mapped to CIM hardware.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Co-design is Crucial for CIM Efficiency: The fundamental insight is that simply applying existing neural network architectures to CIM hardware is suboptimal. A co-designed approach, where architecture search is intertwined with hardware considerations, is necessary to unlock the full potential of CIM for energy efficiency and performance.
  • Mixture-of-Experts for CIM: The paper leverages the Mixture-of-Experts (MoE) paradigm, suggesting that this approach can be effectively adapted for CIM. MoE architectures, by selectively activating specialized "experts," can offer a path to more efficient computation compared to monolithic networks, especially when tailored for CIM.
  • Open Access Accelerates Research: The erratum itself highlights the importance of open access. By making the article freely available, the authors facilitate broader engagement and faster progress in the field of efficient AI hardware.

Key Results

Not discussed in the paper.

Limitations and Future Work

The erratum itself does not detail the limitations or future work of the original article. However, the nature of an erratum implies that the original publication had content that needed correction or clarification, and the correction itself is the primary focus. The original article likely contained specific results and discussions on limitations and future directions that are now accessible through the corrected, open-access version.

Summary updated

BibTeX
@article{han2024erratum,
  author = {Shihao Han and Sishuo Liu and Shucheng Du and Mingzi Li and Zijian Ye and Xiaoxin Xu and Yi Li and Zhongrui Wang and Dashan Shang},
  title = {{Erratum to: CMN: a co-designed neural architecture search for efficient computing-in-memory-based mixture-of-experts}},
  journal = {Science China Information Sciences},
  year = {2024},
  doi = {10.1007/s11432-024-4168-5},
}
Figure