TL;DR
Bayes-Consistent Tempering (BCT) fixes a fundamental mathematical mismatch in how flow-based language models handle temperature scaling. By aligning the model's training distribution and sampling dynamics with the desired temperature, BCT enables these models to finally reap the benefits of low-temperature decoding, leading to significant performance gains in mathematical reasoning and constrained generation tasks.
What Problem Does This Paper Address?
In traditional autoregressive models (like GPT-4), "temperature" is a simple post-hoc knob: you divide the output logits by $T$ to make the model more confident ($T < 1$) or more creative ($T > 1$). However, a new class of models called Flow Language Models (FLMs)—which treat generation as a continuous path from noise to data—don't respond well to this simple trick.
The authors identify two "mismatches" that occur when you apply standard tempering to FLMs:
- Distribution Mismatch: Tempering the final output implicitly changes the distribution of the intermediate "noisy" states. Since the model was trained on untempered noisy states, it encounters "out-of-distribution" data during low-temperature sampling.
- Dynamics Mismatch: FLMs move states toward tokens using a "velocity" field. Standard tempering changes the weights of which token to move toward but uses the original velocity designed for untempered flows.
Because of these inconsistencies, lowering the temperature in FLMs often fails to improve performance on reasoning tasks like GSM8K, whereas it helps significantly in discrete diffusion models.
Methods
The authors introduce Bayes-Consistent Tempering (BCT), which treats temperature as an integral part of the generative flow rather than a post-hoc adjustment. They propose three versions:
- BCT-L (Learned): The "gold standard." The model undergoes Tempered-Flow Training (TFT), where it is explicitly trained on intermediate states drawn from tempered bridges. At inference, it uses a Bayes-corrected posterior and tempered dynamics.
- BCT-P (Path Tempering): A training-free variant for existing models. It keeps the standard tempered posterior but replaces the sampling velocities with those derived for a tempered bridge.
- BCT-E (Endpoint Tempering): Another training-free variant that tempers only the endpoint distribution while keeping the original bridge and dynamics.
The researchers instantiated these methods on two types of flows: Hyperspherical flows (S-FLM), where data lives on the surface of a sphere, and von Mises–Fisher flows (SFCD).
Datasets
The models were evaluated on:
- GSM8K (TinyGSM protocol): A benchmark of grade-school math word problems. The model generates Python code to solve the problems.
- Sudoku-Hard: A task requiring tightly constrained generation where the model must complete a puzzle with a unique valid solution.
Key Insights and Ideas
- Temperature is a Flow Property: The core "aha moment" is that in flow models, temperature isn't just a scaling factor for the final answer; it changes the entire trajectory of how you get there.
- Training-Inference Alignment: BCT-L proves that if you want a model to work well at $T=0.1$, you should probably show it what $T=0.1$ looks like during training. Even a tiny amount of extra training (0.8% of the original pre-training) is enough to see massive gains.
- The "Collapse" of Post-hoc Fixes: While training-free variants like BCT-P help at moderate temperatures, they "collapse" (performance drops to near zero) at very low temperatures ($T=0.1$) because the train-inference mismatch becomes too extreme. Only the learned BCT-L remains robust.
Key Results
The results show that BCT-L consistently outperforms standard tempering across all backbones and tasks.
- GSM8K Accuracy: On the S-arch backbone, standard tempering plateaus around 17-18% accuracy regardless of how low $T$ goes. BCT-L improves steadily as $T$ decreases, reaching 27.0% at $T=0.1$.
- Sudoku-Hard: Standard tempering actually hurts performance as $T$ drops. In contrast, BCT-L improves monotonically, reaching a 46.8% solve rate.
- Robustness: BCT-L is the only method that doesn't degrade under "aggressive" tempering (very low $T$), supporting the idea that temperature should be baked into the generative process.
Limitations and Future Work
- Mathematical Derivations: BCT requires deriving the tempered bridge and corresponding dynamics. While the authors provided these for hyperspherical and von Mises–Fisher flows, extending this to other complex bridge families (like those used in Dirichlet flows) remains future work.
- Retraining Cost: Although the authors showed that BCT-L only requires a small amount of "continual training," it still requires more compute than a purely post-hoc method.
- Generalization: While tested on math and Sudoku, the benefits for open-ended creative writing (where higher temperatures are often preferred) were not the focus of this study.
Summary updated