Implementation of Neural Hardware with the Neural VLSI of URAN in Applications with Reduced Representations

Il Song Han, Ki-Chul Kim, Hwang-Soo LeeNeurIPS 1994

Overview

The URAN (Universally Reconstructible Artificial Neural network) chip bridges the gap between high-speed analog processing and flexible digital interfacing. By utilizing an analog-digital mixed-mode VLSI design, researchers demonstrated that neural hardware can achieve massive throughput while remaining robust enough to handle real-world tasks like speech recognition and servo control, even when numerical precision is significantly reduced.

The Challenge of Early Neural Hardware

In the early 1990s, researchers faced a significant bottleneck: software simulations of neural networks were too slow for real-time applications, while existing digital hardware was often too expensive or power-hungry. Conversely, purely analog neural chips, though fast, suffered from poor accuracy, lack of flexibility, and difficulty in scaling.

The authors address the challenge of creating a "best of both worlds" hardware architecture. They sought to build a system offering the massive parallelism and speed of analog circuits while maintaining the modularity and ease of use found in digital systems. Specifically, they investigated how "reduced representations"—using lower bit-depths for weights and inputs—affect the performance of such hardware in practical tasks.

The URAN Architecture

The core of this research is the URAN chip, a full-custom VLSI circuit fabricated using 1.0µm CMOS technology.

URAN Chip Specifications

Key technical features include:

  • Analog-Digital Mixed-Mode: The chip uses digital interfaces for easy integration but performs the heavy lifting of synaptic multiplication in the analog domain.
  • Voltage-Controlled MOSFET Synapses: Instead of complex digital multipliers, the chip uses the resistance of MOSFET transistors (controlled by voltage) to emulate synaptic weights.
  • Wired-OR Modularity: The synapses act as independent bipolar current sources. Because they use a "wired-OR" interface, multiple chips or modules can be connected without needing complex clocking or synchronous timing, allowing for nearly infinite expansion in network size.
  • Linear Neuron Functions: To simplify the hardware implementation for real-world applications like servo control, the researchers utilized linear neuron functions.
  • Reduced Precision Training: The team trained Multi-Layer Perceptrons (MLPs) on conventional computers using high precision (4-byte) but executed the "recall" or inference phase on the URAN chip using only 1-byte (8-bit) weight accuracy.

Benchmarks and Datasets

The researchers evaluated the URAN hardware using two distinct benchmarks:

  1. Korean Speaker-Independent Digit Recognition: A dataset of 2,000 samples from 20 individuals. Training used 500 samples from 10 people, while testing used 1,000 samples from the remaining 10. Samples were filtered through a 17-channel critical-band filter bank with added noise (SNR from 0dB to clean).
  2. Industrial Servo Control: A model of a digital VCR (Video Cassette Recorder) used to test the chip's ability to handle precise angle control and error reduction in mechanical systems.

Key Insights and Results

One of the most significant findings is that neural networks are surprisingly resilient to "low-fidelity" hardware. Moving from 8-bit (2 decimal) to 4-bit (1 decimal) input accuracy resulted in minimal performance drops. Furthermore, because the chip doesn't require a global clock for its synaptic operations, the hardware scales simply by placing cells in a 2D array, avoiding the timing bottlenecks of large-scale digital circuits. By using simple switches controlled by digital neural inputs to gate analog current sources, the chip achieves a computation speed of 200 Giga Connections Per Second (GCPS).

Speech Recognition Performance

On "clean" audio, the URAN chip achieved 97.5% accuracy with 8-bit precision. Even with binary (1-bit) inputs, it maintained a respectable 90.7% accuracy. In noisy environments (30dB SNR), the system maintained over 96% accuracy (96.2% for 2 decimal). However, performance dropped significantly at 0dB (around 30-38%).

Speech Recognition Accuracy across SNR and Precision Levels

Servo Control and Hardware Density

The URAN-based controller achieved a 70% improvement in error reduction compared to conventional methods, proving that 1-byte weight accuracy is sufficient for industrial control tasks.

In terms of density, a single 13x13 mm² chip could house over 135,000 synapses. The authors proposed a multi-chip module capable of reaching Tera-connections per second—performance comparable to the biological nervous systems of insects.

Limitations and Future Work

While the recall (inference) happens on the URAN chip, the learning (training) is still performed on a conventional computer, creating a dependency on external hardware for the adaptation phase. Additionally, in the servo control application, there was a 30% "non-ideal factor" attributed to limitations in the training data and the hardware's fixed-point nature.

Looking forward, the authors suggested that moving to a 0.5µm CMOS process would allow for a single chip with 1,000,000 connections, further pushing the boundaries of neural hardware density.

Summary updated

BibTeX
@inproceedings{han1994implementation,
  author = {Il Song Han and Ki-Chul Kim and Hwang-Soo Lee},
  title = {{Implementation of Neural Hardware with the Neural VLSI of URAN in Applications with Reduced Representations}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1994},
}
Figure