Deep learning is traditionally a game of high-precision floating-point arithmetic. To calculate a single neuron's output, a processor must multiply many inputs by many weights and sum the results. These multiplications are the primary bottleneck for deploying AI on "edge" devices—like smartwatches, medical implants, or drones—where power and silicon area are strictly limited.
Ternary Neural Networks (TNNs) solve this by restricting weights to just three discrete values: $\{-1, 0, +1\}$. This simple constraint fundamentally changes the nature of computation, turning power-hungry multiplications into simple additions, subtractions, or omissions.
The Intuition: Why Three Values?
In a standard network, a weight can be any real number (e.g., 0.734). In a binary network, weights are restricted to $\{-1, +1\}$. While binary networks are extremely efficient, they lack a "zero" state. As Xiang Deng and Zhongfei Zhang argue in An Embarrassingly Simple Approach to Training Ternary Weight Networks, the zero state is critical because it acts as a form of learned pruning. It allows the network to effectively "turn off" uninformative connections, maintaining higher accuracy than binary models while remaining far more efficient than full-precision ones.
When weights are limited to $\{-1, 0, +1\}$, the mathematical operation of a convolution or a matrix multiplication simplifies:
- If the weight is +1, you simply add the input to the accumulator.
- If the weight is -1, you subtract the input (or flip its sign).
- If the weight is 0, you do nothing.
This "multiplication-free" logic allows researchers like Mohsen Hajabdollahi to implement real-time medical imaging directly inside a Wireless Capsule Endoscopy device, where battery life is the most critical constraint.
Moving Computation to the Sensor
The efficiency of ternary weights is most transformative when computation happens at the "focal plane"—the actual sensor capturing the light. Laurie Bose, Piotr Dudek, and their colleagues demonstrated this with Pixel Processor Arrays (PPAs).
By embedding ternary logic directly into the pixels, they created A Camera That CNNs. Because ternary operations require no dedicated hardware multipliers, the researchers could fit the necessary logic into the tiny spaces between light-sensing elements. This allows the camera to perform object tracking and digit recognition at hundreds of frames per second while consuming less than 2 watts of power.
Scaling to Large Language Models
While early TNN research focused on small vision models, the field has recently pivoted toward Large Language Models (LLMs). Modern LLMs are so large they often cannot fit into the memory of a single GPU. Ternary quantization reduces the memory footprint of these models to approximately 1.58 bits per weight (since $\log_2(3) \approx 1.58$).
Recent frameworks like PT2-LLM, developed by Haotong Qin and Xianglong Yan, show that we can "ternarize" existing LLMs without expensive retraining. By using a small amount of calibration data, they achieve significant inference speedups and memory reductions. Similarly, Shigeng Wang and Anbang Yao introduced CAT-Q, which enables these 1.58-bit models to run on general-purpose CPUs with high throughput, bypassing the need for specialized AI chips.
Bridging the Hardware Gap
A major challenge for TNNs is that most modern hardware (like NVIDIA GPUs) is optimized for binary (1-bit) or 8-bit/16-bit integers, not 1.58-bit "trits." To solve this, researchers use clever encoding schemes.
In Zeros can be Informative, Chunshu Wu and Ang Li describe a "subtractive bit-encoding" scheme. They decompose ternary weights into two binary bit-planes, perform two fast binary operations on standard Tensor Cores, and subtract the results. This allows ternary models to achieve a 2x speedup on existing hardware that wasn't even designed for them.
For future hardware, the potential is even greater. The Kraken SoC, an open-source RISC-V chip designed by Viviane Potocnik and Alfio Di Mauro, includes dedicated ternary accelerators. By replacing multipliers with simple multiplexers and adders, Kraken achieves state-of-the-art energy efficiency for multi-modal perception in wearables.
Summary of the Ternary Advantage
The shift to ternary weights represents a move away from the "brute force" of floating-point math toward a more structural, sparse form of computation. By embracing the $\{-1, 0, +1\}$ set, researchers are:
- Eliminating Multipliers: Reducing the silicon area and energy cost of the most expensive arithmetic operation.
- Enabling Sparsity: Using the "zero" state to prune unnecessary connections automatically.
- Compressing Models: Shrinking massive LLMs to 1.58 bits to fit on edge devices like the Raspberry Pi.
As we move toward a world of "AI everywhere," the ability to perform inference using only additions and subtractions may become the standard for sustainable, ubiquitous intelligence.