ThoughtSource: A central hub for large language model reasoning data

Simon Ott, Konstantin Hebenstreit, Valentin Liévin, Christoffer Egeberg Hother, Milad Moradi, Maximilian Mayrhauser, Robert Praas, Ole Winther, Matthias Samwald

Introduction

Large Language Models (LLMs) like GPT-4 have demonstrated remarkable capabilities, yet they often operate as "black boxes," making it difficult to trace their reasoning or ensure the accuracy of multi-step logic. Chain-of-thought (CoT) prompting—where models are encouraged to verbalize intermediate steps—has become a standard technique to improve transparency and performance. However, CoT data has historically been fragmented across various domains and formats. ThoughtSource addresses this by providing an open-source meta-dataset and software library that standardizes and centralizes reasoning data from 15 diverse benchmarks.

Methodology and Framework

The ThoughtSource framework is built on several pillars to ensure interoperability and ease of use for researchers:

  1. Standardized Schema: By extending the BigBIO project’s schema, the authors created a unified format for question-answering data, allowing disparate datasets to be accessed via a single interface.
  2. Software Library: Integrated with Hugging Face datasets and LangChain, the library facilitates data loading, CoT generation via LLM APIs, and automated evaluation.
  3. Data Transformation: Custom algorithms convert raw rationales (e.g., structured graphs or math expressions) into natural language "reference CoTs."
  4. Interactive Tools: A Dataset Viewer and an Annotator tool allow for manual inspection and error flagging in AI-generated reasoning.

Data Composition

ThoughtSource integrates 15 benchmarks across three primary domains: Scientific/Medical, General-Domain, and Math Word Problems. This breadth ensures that models can be trained and evaluated on a wide variety of reasoning styles, from complex medical case analysis to multi-step arithmetic.

Overview of datasets integrated into ThoughtSource and their respective licenses.

The licensing information provided in Figure 1 highlights the project's commitment to open science, with many datasets available under MIT, Apache, or Creative Commons licenses, facilitating broad academic and commercial use.

Quantitative Analysis and Results

The scale of ThoughtSource is significant, encompassing over 350,000 examples. A key feature of the hub is the inclusion of both human-authored reference CoTs and a massive collection of AI-generated reasoning chains.

Detailed statistics of the datasets within the ThoughtSource hub, including example counts and CoT sources.

As shown in Figure 2, datasets like MedMCQA provide nearly 200,000 examples, while others like MedQA (USMLE) have been augmented with over 135,000 AI-generated CoTs. This dual-source approach allows researchers to compare human logic with machine reasoning. Furthermore, the authors introduced ThoughtSource-33, a curated subset used to study prompt sensitivity by generating 60 unique CoTs per item across various models and strategies.

Key Insights

Analysis of the centralized data revealed several important trends:

  • Data Leakage: N-gram analysis showed significant overlap between datasets (e.g., a 1.0 overlap between WorldTree v2 and EntailmentBank), emphasizing the need for a central hub to ensure rigorous evaluation and prevent models from being tested on training data.
  • Domain Complexity: Reasoning requirements vary significantly by field. Medical datasets (MedQA, PubMedQA) typically require longer questions and more extensive verbal reasoning compared to mathematical word problems.
  • Prompt Sensitivity: The performance and logic of LLMs are highly dependent on specific phrasing (e.g., "Let's think step by step"), a phenomenon the ThoughtSource library is designed to help researchers investigate systematically.

Conclusion and Future Work

ThoughtSource represents a major step toward making LLM reasoning more transparent and reproducible. While AI-generated CoTs currently face challenges with "hallucinations" and faulty logic, the tools provided—such as the Annotator—enable the community to identify and correct these issues. Future efforts will focus on expanding the dataset library and curating "value-aligned" reasoning chains to guide the development of safer, more reliable AI systems.

Summary updated

BibTeX
@inproceedings{ott2023thoughtsource,
  author = {Simon Ott and Konstantin Hebenstreit and Valentin Liévin and Christoffer Egeberg Hother and Milad Moradi and Maximilian Mayrhauser and Robert Praas and Ole Winther and Matthias Samwald},
  title = {{ThoughtSource: A central hub for large language model reasoning data}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41597-023-02433-3},
}
Figure