Signed Weight Representation in Analog Compute in Memory

This direction is split into other directions: signed weight representation. Its last version stays readable and citable here.

Version 1 of 194 papers129 conceptsfrozen 2026-10-04

The fundamental operation of a neural network is the weighted sum: $y = \sum w_i x_i$. In software, this is trivial; weights ($w$) and inputs ($x$) are simply floating-point numbers that can be positive or negative. However, as we move toward Compute-in-Memory (CIM) to break the "von Neumann bottleneck," we encounter a physical wall.

In analog hardware—whether using Resistive RAM (RRAM), Phase-Change Memory (PCM), or even DNA concentrations—the physical properties used to represent weights are almost always unipolar. Conductance cannot be negative; light intensity cannot be negative; chemical concentration cannot be negative. This creates a representational gap: how do we map the bipolar math of neural networks onto the unipolar physics of analog devices?

The Differential Pair Strategy

The most prevalent solution across the literature is the differential mapping strategy. Instead of trying to find a single material that can "subtract," researchers use two physical devices to represent one logical weight.

In this configuration, a weight is defined as the difference between two conductances: $W = G_{pos} - G_{neg}$. Hyungjun Kim and Brian D. Hoskins demonstrate that by using adjacent columns in a crossbar array—one for positive components and one for negative—the net synaptic weight can span the full range of real values. When an input voltage is applied, the currents from these two columns are subtracted at the periphery, effectively realizing a signed multiplication.

This dual-device approach, utilized in IBM’s HERMES chip by Manuel Le Gallo and Riduan Khaddam-Aljameh, does more than just enable signs. It provides a "common-mode" rejection of noise. If temperature or age causes both devices to drift slightly, the difference between them remains relatively stable, mitigating the inherent variability of analog hardware.

Bit Slicing for High Precision

Even after solving the sign problem, analog devices often suffer from limited resolution. A single RRAM cell might only reliably store 2 bits of information, yet deep learning models often require 8-bit or even 16-bit precision.

To bridge this, researchers employ bit slicing or decomposition. Anand Raghunathan and Kaushik Roy describe distributing a high-precision weight across multiple low-precision cells. For instance, a 16-bit weight can be split into two 8-bit synapses: one representing the Most Significant Byte (MSB) and another the Least Significant Byte (LSB). Bradley H. Theilman demonstrated this on Intel’s Loihi 2, showing that parallel synapses can be summed to reconstruct high-fidelity values, though this requires a careful balance of hardware resources.

Alternative Physical Mappings

While differential pairs are the standard, other researchers have explored more compact or domain-specific ways to handle signs:

  • Reference Partitioning: In early neuromorphic work, Alister Hamilton proposed partitioning a voltage range around a central reference point. Deviations above the threshold are excitatory; deviations below are inhibitory. This reduces the physical footprint by avoiding the need for two separate pathways.
  • Inverter Circuits: Michael Beigl explored using an inverter to flip the input voltage polarity before it enters the crossbar. This allows for subtractive signals in analog multiply-accumulate operations without requiring specialized "negative" resistors.
  • Molecular Annihilation: In the realm of DNA computing, Xiao Liu implemented negative weights through a physical annihilation mechanism. Specific DNA duplexes consume strands representing positive values, performing chemical subtraction that mimics the behavior of inhibitory synapses.

Software-Hardware Co-design

Perhaps the most elegant way to handle the limitations of analog hardware is to change the software to fit the physics.

Geng Yuan proposed a training-time regularization that enforces sign consistency. By forcing all weights in a specific hardware column to share the same sign during training, the system eliminates the need for complex differential circuits entirely.

Furthermore, training in-situ (directly on the hardware) allows the network to "learn" the specific non-linearities and defects of the physical devices. Tayfun Gokmen introduced the Tiki-Taka algorithm, a dual-matrix approach that separates noisy gradient accumulation from stable weight updates. This allows analog systems with asymmetric switching—where it's easier to increase conductance than decrease it—to match the accuracy of high-precision digital models.

By combining these representational strategies, researchers are transforming "imperfect" physical materials into high-performance neural engines, proving that the constraints of physics can be overcome through clever architectural and algorithmic mapping.

Papers in this version

Showing 60 of 94 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_signed_weight_representation_in_analog_c_0e083c72,
  title = {Signed Weight Representation in Analog Compute in Memory},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/signed-weight-representation-in-analog-compute-in-memory/rdr_0e083c72424941a0aecdc5be570d39dc},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_signed_weight_representation_in_analog_c_0e083c72_v1,
  title = {Signed Weight Representation in Analog Compute in Memory},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/signed-weight-representation-in-analog-compute-in-memory/rdr_0e083c72424941a0aecdc5be570d39dc/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}