TL;DR
ThoughtSource is an open-source meta-dataset and software framework designed to standardize, evaluate, and annotate Chain-of-Thought (CoT) reasoning data for large language models (LLMs). By centralizing 15 popular reasoning benchmarks across medicine, math, and general knowledge into a unified schema, it provides researchers with standardized tools for prompt execution, multi-model evaluation, and interactive error analysis.
What Problem Does This Paper Address?
While LLMs frequently achieve strong benchmark results, they struggle with multi-hop reasoning, prone to hallucinations, and act as opaque black boxes. Eliciting reasoning via Chain-of-Thought (CoT) prompting substantially mitigates these flaws, but the surrounding research ecosystem is fragmented:
- Datasets format rationales, entailment trees, and mathematical formulas in vastly incompatible schemas.
- Prompting strategies (e.g., zero-shot vs. few-shot triggers) and answer-extraction pipelines are inconsistently implemented across papers.
- Qualitative evaluation of reasoning steps remains tedious, lacking unified tools to inspect, contrast, and annotate generated rationales across different models.
Methods
ThoughtSource introduces a unified data and software architecture centered around three components:
- Standardized Schemas: The authors designed a unified ThoughtSource schema (extending the BigBIO question-answering format) alongside original source schemas. Disparate explanation formats—such as nested algebraic operations or structured entailment trees—are systematically converted into coherent natural language reasoning steps.
- Library & Execution Pipeline: Built on top of Hugging Face
datasetsand LangChain, the library standardizes dataset loading, prompt triggering (e.g., modularized reasoning instructions and extraction phrases), API calling across model providers, and automated answer evaluation. - Web-Based Inspection & Annotation Tools: The platform includes an interactive web interface (The ThoughtSource Annotator) that highlights textual similarities across multiple CoTs using adjustable TF-IDF thresholds, enabling researchers to identify where reasoning paths diverge, hallucinate, or fail.
Datasets
The initial release integrates 15 benchmarks across three reasoning domains:
- Scientific and Medical QA (7): WorldTree V2, EntailmentBank, OpenBookQA, MedQA (USMLE), MedMCQA, PubMedQA, and the medical subsets of MMLU.
- General-Domain QA (3): CommonsenseQA (including ECQA rationales), StrategyQA, and QED.
- Math Word Problems (5): AQUA-RAT, ASDiv, GSM8K, MAWPS, and SVAMP.
Additionally, the authors provide ThoughtSource-33, a curated diagnostic benchmark consisting of 33 samples from six diverse datasets (198 problems total). For each problem, 60 unique zero-shot reasoning chains were generated by combining six foundational models (GPT-4, GPT-3.5-turbo, text-davinci-002, text-davinci-003, Flan-T5-XXL, and Cohere command-xlarge-nightly) with ten distinct prompting strategies.
Key Insights and Ideas
- Significant Benchmark Overlap: Analysis using Szymkiewicz–Simpson overlap coefficients revealed hidden redundancies between widely used benchmarks. For example, EntailmentBank questions have a 1.0 (100%) overlap with WorldTree V2, and math benchmarks like ASDiv, MAWPS, and SVAMP share substantial question and template overlap because of common seed data or shared web scrapes.
- Unified Linearization of Non-Linear Logic: The paper demonstrates that complex non-linear explanations—such as directed entailment trees or abstract syntax trees of arithmetic operations—can be converted into natural language CoTs via rule-based verbalization without losing deductive flow.
- Comparative Error Localization: By visually aligning generated chains across diverse model families and prompt triggers, the authors highlight that errors often cluster around specific multi-hop transition steps rather than final arithmetic or extraction phases.
Key Results
- Scale: Unifies hundreds of thousands of examples, including massive domain collections like MedMCQA (193,155 questions) and AQUA-RAT (97,975 questions).
- Enriched Reasoning Data: Incorporates existing and newly synthesized AI-generated reasoning chains, such as over 135,000 generated CoTs for MedQA and over 106,000 for MedMCQA.
- Length Distributions: Statistical profiling shows substantial variation across tasks; MedQA presents the highest median question length, while PubMedQA contains the longest median CoT length. Context fields are explicitly isolated for PubMedQA and QED (averaging 116 and 56 tokens, respectively).
Limitations and Future Work
- Noise in Generated CoTs: The repository ingests zero-shot and few-shot outputs from current LLMs, which inevitably contain logical fallacies, arithmetic slips, and hallucinations.
- Linear Reasoning Focus: The current schema focuses on linear chains of thought; branching search structures (e.g., Tree-of-Thoughts or Graph-of-Thoughts) are not natively modeled.
- Context Availability: Rich context fields are currently populated for only a subset of benchmarks (PubMedQA and QED).
- Future Directions: Expanding the hub through community contributions to include additional domains, non-English datasets, and downstream fine-tuning/RLHF pipelines utilizing the annotator tool.
Summary updated