TL;DR
Stream of Belief (SoB) is a novel multi-agent reasoning framework that replaces slow, discrete natural language dialogue with continuous latent "belief tokens." By treating collective reasoning as a dynamical system seeking a stable equilibrium, SoB achieves significantly higher accuracy while slashing token usage by half and doubling inference speed compared to traditional methods.
What Problem Does This Paper Address?
In the current era of "Agentic AI," we often have multiple LLM agents collaborate to solve complex problems. Traditionally, these agents talk to each other using natural language—exchanging critiques and reasoning traces. While intuitive, this is incredibly inefficient: it's slow, consumes massive amounts of tokens, and forces the model's rich, continuous internal representations into a "lossy" discrete format (words).
The authors ask: Can collective intelligence emerge if agents interact directly through their latent beliefs? The challenge lies in how to represent these beliefs and how to update them iteratively so that a group of agents can converge on a correct answer without the overhead of full-text generation at every step.
Methods
The core of the paper is the Stream of Belief (SoB) framework. Instead of generating text, agents generate Belief Tokens—continuous vectors formed by a probability-weighted mixture of the model's vocabulary embeddings.
Reasoning proceeds in synchronous cycles consisting of two operations:
- Global Interaction: Each agent updates its belief by looking at the problem and the current latent beliefs of all other agents. This allows information to propagate across the system.
- Local Refinement: Each agent independently "thinks" about its own belief to refine it further.
The system iterates until it reaches a collective equilibrium—a fixed point where further communication doesn't meaningfully change the agents' internal states. Only then is a final answer decoded into text.
Datasets
The authors evaluated SoB across eight diverse benchmarks covering math, science, logic, and multi-hop reasoning:
- Math: GSM8K, MATH-500.
- Science/Knowledge: MMLU-Pro, GPQA-Diamond, ARC-Challenge, MedQA.
- Logic/Reasoning: BIG-Bench Hard (BBH), HotpotQA.
They tested these on a wide range of model families, including Qwen3, Llama-3.1, Gemma-3, and Phi-4-mini, scaling from 0.6B up to 405B parameters.
Key Insights and Ideas
-
Reasoning as a Landscape: The authors visualize reasoning as navigating a "belief landscape." Individual agents might get stuck in "wrong basins" (incorrect answers). Global interaction reshapes this landscape, strengthening the "correct basin" and helping agents escape local minima. Conceptual view of how interaction reshapes the belief landscape to favor correct solutions.
-
Spontaneous Sharpening: A fascinating "aha moment" is that even though belief tokens start as fuzzy mixtures of many words, they naturally "sharpen" over time. As the system reaches equilibrium, these latent vectors spontaneously align with specific, discrete vocabulary embeddings.
-
Equilibrium $\neq$ Consensus: Collective intelligence doesn't require everyone to agree. Agents can reach a stable global state while still maintaining heterogeneous beliefs (e.g., a 3:1 majority), as shown in their graph-connectivity experiments. A qualitative example showing agents reaching a stable 3:1 equilibrium on a graph problem.
-
Parallel Efficiency: Because SoB distributes computation across parallel belief trajectories rather than extending a single long sequential chain, it is fundamentally more efficient than sequential latent reasoning.
Key Results
- Superior Accuracy: SoB consistently outperformed other latent reasoning baselines, improving accuracy by up to 15.7 points. It even remained competitive with (and sometimes beat) full-text multi-agent debate.
- Massive Efficiency Gains: On average, SoB used 51% fewer text tokens (and up to 82% fewer on some tasks) compared to the most efficient baselines.
-
Faster Inference: It delivered 1.1x to 2.6x lower token latency, proving that latent communication is a much faster medium for collaboration than text. Latency and token usage comparisons showing SoB's significant efficiency advantages.
-
Robust Scaling: The benefits of SoB hold true as models get larger, suggesting that latent collaboration is a viable strategy for scaling up the reasoning capabilities of the next generation of LLMs. Scaling curves showing SoB's consistent performance across different model sizes.
Limitations and Future Work
The authors acknowledge several limitations:
- Interpretability: Because the "thinking" happens in latent space, it is harder for humans to inspect the intermediate reasoning steps compared to a text-based Chain-of-Thought.
- Fixed Capacity: The current framework uses fixed-length belief representations, which might constrain the complexity of information that can be exchanged for extremely difficult problems.
- Frozen Models: The study focused on frozen models. Future work could explore how fine-tuning models specifically for latent belief dynamics might further unlock collective intelligence.
Overall, Stream of Belief offers a compelling new perspective: perhaps the best way for AI agents to work together isn't by talking, but by "feeling" out a collective equilibrium in the high-dimensional space of their own thoughts.
Summary updated