Decentralized Consensus under Quantized Communication Constraints

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2109 papers156 conceptsfrozen 2026-10-04

In a perfect world, distributed agents—whether they are sensors in a power grid, GPUs in a data center, or autonomous underwater vehicles—would share information with infinite precision. In reality, communication is bounded by the physical limits of hardware and the high cost of bandwidth. To operate within these limits, agents must quantize their data, converting continuous real numbers into a finite number of bits.

While quantization solves the bandwidth problem, it introduces a new one: quantization noise. In a decentralized network, this noise does not stay local. It propagates, accumulates, and can ultimately prevent the network from ever reaching a true consensus.

The Problem of Error Accumulation

In centralized systems, a parameter server can often manage compression errors because it sees the global state. However, as Hanlin Tang and Tong Zhang point out in their work on DeepSqueeze, decentralized peer-to-peer topologies lack this central authority. When an agent rounds a value to fit a bit budget, that "rounding error" is lost to its neighbors. Over many iterations, these small discrepancies compound across the network graph.

This accumulation leads to several failure modes:

  1. The Error Floor: The system approaches the optimal solution but begins to oscillate or "chatter" within a neighborhood of the goal, unable to reach it exactly due to persistent noise Aghasi et al..
  2. Systematic Bias: Simple rounding schemes (like always rounding down) can cause the entire network to converge to a value significantly lower than the true global average Nedić et al..
  3. Divergence: In non-convex optimization or high-stakes control loops, such as those for underwater vehicles Yang et al., quantization noise can destabilize the entire system, leading to a total loss of synchronization.

Building Intuition: Three Strategies for Precision

Researchers have developed three primary mechanisms to allow low-precision communication to achieve high-precision results.

1. Dithering: Breaking Deterministic Error

If quantization error is deterministic, it creates predictable biases. José M. F. Moura demonstrated that by adding a small amount of random noise—dither—before quantization, the resulting error becomes zero-mean noise Distributed Consensus Algorithms in Sensor Networks: Quantized Data. This ensures that while individual messages are "noisy," the network's long-term average remains unbiased, allowing sensors to eventually converge to the true parameter.

2. Error Feedback: Remembering the Remainder

The most successful modern algorithms, such as CHOCO-SGD Koloskova et al. and DEF-ATC Nassif et al., use a "memory" mechanism. When an agent quantizes a vector, it calculates the difference between the precise value and the quantized version. Instead of discarding this residual, the agent stores it locally and adds it to the next update. This ensures that no information is permanently lost; it is merely delayed. This "error compensation" allows decentralized training to match the convergence rates of full-precision systems while using up to 90% fewer bits Tang et al..

3. Adaptive Quantization: Zooming In

As a network nears consensus, the differences between agents' values become smaller. If the quantization range remains fixed, the relative noise becomes overwhelming. Siva Theja Maguluri and Justin Romberg proposed adaptive quantization, where agents dynamically shrink their quantization range as they converge. By "zooming in" on the shrinking error, the network can maintain a constant signal-to-noise ratio, achieving the same fast convergence rates as infinite-precision communication.

Convergence and Network Topology

The speed at which a quantized network reaches agreement depends heavily on its "connectivity." Early work often relied on exponential bounds, but Angelia Nedić, Alex Olshevsky, and John N. Tsitsiklis established tighter polynomial bounds, proving that consensus is achievable even in time-varying networks where links frequently fail.

Further refinements by Shang Shang and Paul W. Cuff used analogies from electric network theory to show that for any static graph, quantized consensus is reached in $O(N^3 \log N)$ time An Upper Bound on the Convergence Time for Quantized Consensus of Arbitrary Static Graphs. This theoretical foundation confirms that decentralized systems are robust: as long as the network remains connected over time, quantization noise can be managed, and a global agreement can be reached.

Papers in this version

Showing 60 of 109 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 87 papers
  2. Version 1 · 2026-10-04 · 109 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_quantized_communication_21d8e31f,
  title = {quantized communication},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/quantized-communication/rdr_21d8e31ff74c4d0db3377f99858ec17a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_decentralized_consensus_under_quantized__21d8e31f_v1,
  title = {Decentralized Consensus under Quantized Communication Constraints},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decentralized-consensus-under-quantized-communication-constraints/rdr_21d8e31ff74c4d0db3377f99858ec17a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}