The fundamental operation of a neural network is the weighted sum: $y = \sum w_i x_i$. In software, this is trivial; weights ($w$) and inputs ($x$) are simply floating-point numbers that can be positive or negative. However, as we move toward Compute-in-Memory (CIM) to break the "von Neumann bottleneck," we encounter a physical wall.
In analog hardware—whether using Resistive RAM (RRAM), Phase-Change Memory (PCM), or even DNA concentrations—the physical properties used to represent weights are almost always unipolar. Conductance cannot be negative; light intensity cannot be negative; chemical concentration cannot be negative. This creates a representational gap: how do we map the bipolar math of neural networks onto the unipolar physics of analog devices?
The Differential Pair Strategy
The most prevalent solution across the literature is the differential mapping strategy. Instead of trying to find a single material that can "subtract," researchers use two physical devices to represent one logical weight.
In this configuration, a weight is defined as the difference between two conductances: $W = G_{pos} - G_{neg}$. Hyungjun Kim and Brian D. Hoskins demonstrate that by using adjacent columns in a crossbar array—one for positive components and one for negative—the net synaptic weight can span the full range of real values. When an input voltage is applied, the currents from these two columns are subtracted at the periphery, effectively realizing a signed multiplication.
This dual-device approach, utilized in IBM’s HERMES chip by Manuel Le Gallo and Riduan Khaddam-Aljameh, does more than just enable signs. It provides a "common-mode" rejection of noise. If temperature or age causes both devices to drift slightly, the difference between them remains relatively stable, mitigating the inherent variability of analog hardware.
Bit Slicing for High Precision
Even after solving the sign problem, analog devices often suffer from limited resolution. A single RRAM cell might only reliably store 2 bits of information, yet deep learning models often require 8-bit or even 16-bit precision.
To bridge this, researchers employ bit slicing or decomposition. Anand Raghunathan and Kaushik Roy describe distributing a high-precision weight across multiple low-precision cells. For instance, a 16-bit weight can be split into two 8-bit synapses: one representing the Most Significant Byte (MSB) and another the Least Significant Byte (LSB). Bradley H. Theilman demonstrated this on Intel’s Loihi 2, showing that parallel synapses can be summed to reconstruct high-fidelity values, though this requires a careful balance of hardware resources.
Alternative Physical Mappings
While differential pairs are the standard, other researchers have explored more compact or domain-specific ways to handle signs:
- Reference Partitioning: In early neuromorphic work, Alister Hamilton proposed partitioning a voltage range around a central reference point. Deviations above the threshold are excitatory; deviations below are inhibitory. This reduces the physical footprint by avoiding the need for two separate pathways.
- Inverter Circuits: Michael Beigl explored using an inverter to flip the input voltage polarity before it enters the crossbar. This allows for subtractive signals in analog multiply-accumulate operations without requiring specialized "negative" resistors.
- Molecular Annihilation: In the realm of DNA computing, Xiao Liu implemented negative weights through a physical annihilation mechanism. Specific DNA duplexes consume strands representing positive values, performing chemical subtraction that mimics the behavior of inhibitory synapses.
Software-Hardware Co-design
Perhaps the most elegant way to handle the limitations of analog hardware is to change the software to fit the physics.
Geng Yuan proposed a training-time regularization that enforces sign consistency. By forcing all weights in a specific hardware column to share the same sign during training, the system eliminates the need for complex differential circuits entirely.
Furthermore, training in-situ (directly on the hardware) allows the network to "learn" the specific non-linearities and defects of the physical devices. Tayfun Gokmen introduced the Tiki-Taka algorithm, a dual-matrix approach that separates noisy gradient accumulation from stable weight updates. This allows analog systems with asymmetric switching—where it's easier to increase conductance than decrease it—to match the accuracy of high-precision digital models.
By combining these representational strategies, researchers are transforming "imperfect" physical materials into high-performance neural engines, proving that the constraints of physics can be overcome through clever architectural and algorithmic mapping.