An Analog Neural Network Inspired by Fractal Block Coding

Fernando J. Pineda, Andreas G. AndreouNeurIPS 1994

Overview

Researchers at Johns Hopkins University have developed a novel approach to fractal image decompression by leveraging the "free" physics of analog CMOS circuits. By mapping a linear mathematical problem onto the naturally nonlinear dynamics of subthreshold transistors, the team created a 16-neuron chip that converges to a stable, fractal-like state with extremely low power consumption (just 41 $\mu$W).

The Challenge of Fractal Block Coding

Fractal block coding is a compression technique that represents an image as a set of self-similar transformations. To decompress the image, one starts with a random input and iteratively applies these transformations until the system converges to a "fixed point"—the original image.

On traditional digital computers, this process is computationally expensive, requiring numerous iterations of matrix multiplication to solve high-dimensional, sparse linear equations. The authors propose that instead of forcing digital hardware to simulate these iterations, we can build a physical dynamical system—an analog circuit—whose natural equilibrium is the decompressed image. The primary challenge is bridging the gap between the highly nonlinear nature of real-world transistors and the linear mathematics of fractal coding.

Methods: Exploiting Nonlinearity in Silicon

The researchers designed a recurrent neural network using subthreshold CMOS VLSI technology. Rather than trying to "fix" the exponential transfer characteristics of transistors to make them linear, the authors embraced them.

The Analog Neuron

Each "neuron" in their circuit consists of three distinct sections:

  1. Input Section: Converts voltage to current and multiplies it by a weight ($a$).
  2. Bias Section: Adds or subtracts a specific current ($b$).
  3. Output Section: Converts the resulting current back into a logarithmically coded voltage.
Circuit diagram of the analog neuron showing input, bias, and output sections

By deriving a specific mapping, the team could set external control voltages ($V_a$ and $V_b$) such that the nonlinear system’s unique stable fixed point exactly matches the solution of the desired linear system. To ensure the network wouldn't oscillate or behave chaotically, they employed the "even loop criterion." By ensuring any feedback loops in the circuit passed through an even number of inhibitory weights, they guaranteed global asymptotic stability.

Experimental Setup

Instead of standard machine learning datasets like MNIST, this study utilizes a "toy" fractal transformation designed to generate a self-similar curve (a fractal ramp). The data consists of the 16-dimensional state vectors representing the output of the 16 neurons on their test chip.

Key Insights

  • Physics as Computation: The core insight is that nonlinear approaches can sometimes be the simplest way to solve linear problems. If the natural operations of the hardware (like subthreshold transistors in silicon) are nonlinear, it is computationally "cheaper" to adapt the math to the hardware rather than forcing the hardware to act linearly.
  • Programming the Equilibrium: The network does not need to be "trained" in the traditional sense. Instead, it is "programmed" by calculating the specific voltages needed to shift the system's equilibrium point to the desired decoded state.
  • Spurious Solution Suppression: Mathematical proofs in the paper show that by choosing specific bias parameters ($b > 0$), the "desired" solution becomes stable while all "spurious" (zero-output) solutions are rendered unstable.

Key Results

The team fabricated a prototype chip with 16 neurons using a 2$\mu$m p-well CMOS process. The results demonstrated significant advantages in efficiency and speed:

  • Ultra-Low Power: Operating at 2.5V, the chip drew only 16.3 $\mu$A, resulting in a total power dissipation of just 41 $\mu$W.
  • High Speed: Both simulations and experimental observations indicated the chip settles into its final state in less than one millisecond.
  • Qualitative Success: The chip successfully generated "qualitative fractals"—outputs that exhibited the self-similar features expected from the mathematical model.
Output graph showing the 16-neuron state vector forming a qualitative fractal ramp

Limitations and Future Work

While the prototype successfully demonstrated the concept, the authors noted that several layout errors on the first chip prevented precise quantitative measurements. They were able to verify stability and qualitative shapes, but not exact numerical accuracy.

Moving forward, the 16-neuron system serves as a foundational proof-of-concept. The authors have already fabricated a larger 32-pixel design with more flexible connection topologies. The ultimate goal is to scale this hardware to decompress actual complex block-coded images, which will require significantly more neurons and a more complex interconnect fabric.

Summary updated

BibTeX
@inproceedings{pineda1994analog,
  author = {Fernando J. Pineda and Andreas G. Andreou},
  title = {{An Analog Neural Network Inspired by Fractal Block Coding}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1994},
}
Figure