Abstract
Large Language Models (LLMs) have recently achieved impressive results in complex reasoning tasks through Chain of Thought (CoT) prompting. However, most existing CoT methods rely on using the same prompts, whether manually designed or automatically generated, to handle the entire dataset. This one-size-fits-all approach may fail to meet the specific needs arising from the diversities within a single dataset. To solve this problem, we propose the Clustered Distance-Weighted Chain of Thought (CDW-CoT) method, which dynamically constructs prompts tailored to the characteristics of each data instance by integrating clustering and prompt optimization techniques. Our method employs clustering algorithms to categorize the dataset into distinct groups, from which a candidate pool of prompts is selected to reflect the inherent diversity within the dataset. For each cluster, CDW-CoT trains the optimal prompt probability distribution tailored to their specific characteristics. Finally, it dynamically constructs a unique prompt probability distribution for each test instance, based on its proximity to cluster centers, from which prompts are selected for reasoning. CDW-CoT consistently outperforms traditional CoT methods across six datasets, including commonsense, symbolic, and mathematical reasoning tasks. Specifically, when compared to manual CoT, CDW-CoT achieves an average accuracy improvement of 25.34% on LLaMA2 (13B) and 15.72% on LLaMA3 (8B).
Introduction
Chain-of-Thought (CoT) prompting has revolutionized how Large Language Models (LLMs) handle complex reasoning by asking them to "think step-by-step." However, most current methods (whether manual or automated) use a static set of prompts for an entire dataset. This "one-size-fits-all" approach ignores the inherent diversity within datasets. For example, a math dataset might contain both simple arithmetic and complex word problems; using the same reasoning exemplars for both is sub-optimal.
CDW-CoT addresses this lack of granularity by creating a "soft" assignment: it calculates a unique prompt strategy for every single test instance based on its similarity to various data clusters, moving away from the brittle "hard" assignment of traditional clustering methods.
Methodology
The CDW-CoT framework operates in three distinct phases to move from global to instance-specific prompting.
1. Cluster-Based Prompt Candidate Pool Initialization
The training dataset is converted into vector embeddings using a sentence transformer and grouped into $K$ clusters via K-means. Representative "candidate prompts" are then selected from each cluster and refined using Zero-Shot-CoT to create a diverse pool that reflects the dataset's inherent variety.
2. Optimizing Prompt Probability Distributions
Instead of just picking the "best" prompt, the system uses Black-Box Prompt Learning (BBPL). It treats prompt selection as a reinforcement learning problem, optimizing a probability distribution ($p^{(i)}$) for each cluster. This ensures that the prompts most effective for a specific category of questions are given higher weights.
3. Distance-Weighted Prompt Selection and Inference
When a new test question arrives, the system calculates its Euclidean distance to every cluster center. These distances are converted into weights using a temperature-scaled softmax function. The final prompt used for reasoning is selected from a unique probability distribution tailored specifically for that instance—a weighted blend of the optimized distributions from all clusters.
Experimental Evaluation
The authors evaluated CDW-CoT on six benchmark datasets across three reasoning domains: Commonsense Reasoning (CSQA, StrategyQA), Symbolic Reasoning (Last Letter Concatenation, Coin Flip), and Mathematical Reasoning (MultiArith, AQuA).
Key Results
The performance gains are substantial, particularly on mid-sized models:
- LLaMA2 (13B): CDW-CoT achieved an average accuracy improvement of 25.34% over Manual-CoT. On the "Letter" symbolic reasoning task, accuracy jumped from 15.80% to 82.67%.
- LLaMA3 (8B): The method showed a 15.72% average improvement over Manual-CoT, proving that even highly capable newer models benefit from tailored prompting.
As shown in the figure above, CDW-CoT consistently outperforms the best baselines across various temperature settings. The temperature parameter $T$ controls how much weight is given to distant clusters, with $T=0.3$ generally providing the best balance between cluster specificity and global context.
Conclusion and Future Work
CDW-CoT demonstrates that "soft" assignment via distance-weighting is superior to "hard" clustering for prompt selection. By forcing the model to develop cluster-specific distributions, the framework effectively "mines" more effective reasoning paths tailored to the complexity of individual questions. While the initial optimization phase requires pre-computation, the resulting accuracy gains are significant. Future work will involve extending CDW-CoT to multimodal tasks, such as image-text reasoning, where visual clusters could inform dynamic reasoning chains.
Summary updated