Automated Atomic Silicon Quantum Dot Circuit Design via Deep Reinforcement Learning

Robert Lupoiu, Samuel S. H. Ng, Jonathan A. Fan, Konrad WalusarXiv 2022 · DOI · arXiv

Abstract

As traditional CMOS scaling approaches its physical limits, Silicon Dangling Bonds (SiDBs) have emerged as a promising candidate for "beyond-CMOS" electronics. However, the design of SiDB circuits is exceptionally challenging due to complex, non-intuitive Coulombic interactions and a discrete, sparse-reward design space. This paper introduces the first automated design tool for SiDB quantum dot circuits using a "tabula rasa" Double-Deep Q-Learning (DDQN) approach. The reinforcement learning agent successfully designs complex logic gates, including a half-adder, that are an order of magnitude more compact and created orders of magnitude faster than previous human-designed benchmarks.

Introduction: The Challenge of Atomic-Scale Design

Traditional CMOS scaling is reaching its fundamental physical limits. Silicon Dangling Bonds (SiDBs) are atomic-scale quantum dots created by removing individual hydrogen atoms from a silicon surface. While they offer extreme miniaturization and flexibility, designing functional circuits with them is a significant hurdle. Unlike traditional electronics, SiDB logic is governed by screened Coulombic interactions between atoms rather than predefined unit cells.

The design space is "hyperdimensional" and lacks gradients; a layout either satisfies a truth table or it does not. This means rewards are sparse, and a single misplaced atom can render an entire circuit useless. Historically, this has necessitated a slow, manual process of trial and error using CAD tools like SiQAD.

Methods: Reinforcement Learning for Circuit Layout

To overcome these challenges, the authors framed circuit design as a reinforcement learning (RL) problem. They utilized a Double-Deep Q-Learning (DDQN) algorithm to navigate the complex design space without prior knowledge of semiconductor physics.

Reinforcement Learning framework for SiDB circuit design, featuring a CNN-based agent and an epsilon-greedy policy.

The Agent and Environment

Starting with an empty design area between input and output wires, the agent iteratively places SiDB dots on a lattice. The neural network architecture consists of three convolutional layers to capture spatial relationships between dots, followed by three fully connected layers.

The system employs an epsilon-greedy policy, starting with 100% exploration to learn the environment and gradually annealing to a 10% exploration rate as it gains experience. To address the sparse reward problem, a shaped reward function was implemented:

  • Small negative rewards for each placement step to encourage efficiency.
  • Partial rewards for satisfying individual rows of a truth table.
  • A large positive reward (+1.0) for discovering a completely new, working layout.

The "data" for training is generated through interaction with a simulated environment based on SiQAD, a physically calibrated simulation tool for SiDB layouts.

Key Results and Insights

The RL agent was evaluated on its ability to generate layouts for standard logic gates (AND, OR, NAND, XOR, XNOR) and a complex two-input, two-output half-adder.

Performance and Efficiency

The AI-designed half-adder represents a significant breakthrough, using an order of magnitude fewer SiDBs than the only other half-adder previously documented in the literature. Furthermore, the automated design process was several orders of magnitude faster than manual design.

Average reward over training epochs for the half-adder circuit, comparing the RL agent against a random baseline.

Policy Convergence and Robustness

The agent demonstrated "robust policy convergence," consistently finding optimal design strategies regardless of the random seed used for training. Interestingly, the AI converged on strategies similar to those used by human experts—such as placing dots symmetrically—confirming these as optimal physical configurations. Because the AI is not biased by traditional human design conventions, it often discovers more compact atomic arrangements than a human designer would consider.

Conclusion and Future Directions

This work has already led to the creation of the Bestagon gate library, a standardized collection of SiDB logic gates that provides a foundation for higher-level automation.

While a major leap forward, future work must address physical variability on real-world silicon surfaces, such as defects and varying dopant concentrations. Additionally, scaling this approach to larger, multi-gate integrated circuits may require hierarchical design methods or more advanced RL architectures. The next critical step is the physical fabrication and experimental validation of these AI-generated layouts.

Summary updated

Abstract

Robust automated design tools are crucial for the proliferation of any computing technology. We introduce the first automated design tool for the silicon dangling bond quantum dot computing technology, which is an extremely versatile and flexible single-atom computing circuitry framework. The automated designer is capable of navigating the complex, hyperdimensional design spaces of arbitrarily sized design areas and truth tables by employing a tabula rasa double-deep Q-learning reinforcement learning algorithm. Robust policy convergence is demonstrated for a wide range of two-input, one-output logic circuits and a two-input, two-output half-adder, designed with an order of magnitude fewer SiDBs in several orders of magnitude less time than the only other half-adder demonstrated in the literature. We anticipate that reinforcement learning-based automated design tools will accelerate the development of the SiDB quantum dot computing technology, leading to its eventual adoption in specialized computing hardware.
BibTeX
@misc{lupoiu2022automated,
  author = {Robert Lupoiu and Samuel S. H. Ng and Jonathan A. Fan and Konrad Walus},
  title = {{Automated Atomic Silicon Quantum Dot Circuit Design via Deep Reinforcement Learning}},
  year = {2022},
  eprint = {2204.06288},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.06288},
}
Figure