Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations

Md. Arafat Sultan, Jatin Ganhotra, Ramón Fernandez Astudillo

Abstract

We introduce a structured chain-of-thought (SCoT) prompting approach to generating content-grounded multi-turn question-answer conversations with a pre-trained large language model (LLM). At the core of our proposal is a structured breakdown of the complex task into a number of states in a state machine, so that actions corresponding to various subtasks, e.g., content reading and utterance generation, can be executed in their own dedicated states. Each state leverages a unique set of resources, including prompts and (optionally) additional tools, to augment the generation process. Automatic evaluation shows that SCoT prompting with designated states for hallucination mitigation can increase agent faithfulness to grounding documents by up to 16.8%. When used as training data, our open-domain conversations synthesized from only 6 Wikipedia-based seed demonstrations train strong conversational QA agents. In out-of-domain evaluation, for example, we observe improvements of up to 13.9% in F1-score against ground truth over target domain gold data when the latter is augmented with our generated examples.

Introduction

Large Language Models (LLMs) are notorious for "hallucinating"—generating facts that aren't in the provided source text. This is particularly problematic in closed-domain conversational QA, where an agent must answer questions based strictly on a specific document. Multi-turn conversations exacerbate this: as the dialogue progresses, the model is more likely to lose track of the source text and rely on its internal (and potentially incorrect) memory. Furthermore, creating high-quality, human-annotated data for these tasks is incredibly expensive and slow.

Researchers at IBM have introduced Structured Chain-of-Thought (SCoT), a prompting strategy that breaks down the complex task of generating grounded multi-turn conversations into a modular state machine. By separating "reading" tasks (like checking if a question is answerable) from "writing" tasks, they aim to reduce hallucinations and produce high-quality synthetic training data using only a handful of "seed" examples.

Methodology

The core innovation is the SCoT approach. Instead of asking an LLM to generate a whole conversation turn in one go, the process is decomposed into a state machine with four distinct actions:

  1. User Utterance Generation (uu): Generating a plausible next question from a user.
  2. Question Answerability Classification (ac): Determining if the provided document actually contains the answer to that question.
  3. Answer Sentence Selection (ss): Identifying the specific sentences in the document that contain the answer.
  4. Agent Utterance Generation (au): Formulating the final response based on the selected sentences.
SCoT state machine architecture for grounded conversation generation.

This modularity allows for LM Augmentation. For example, while a base model (like Falcon-40B) might generate the dialogue, a specialized instruction-tuned "assistant" model (like FLAN-UL2-20B) can be plugged into the ac and ss states to handle the rigorous reading and logic tasks.

Results and Analysis

The authors started with a tiny "seed" set of 6 manually written Wikipedia-based conversations. Using the SCoT method, they scaled this to a synthetic dataset of 1,000 conversations. Evaluation on benchmarks such as DoQA and QuAC yielded several key insights:

  • Modularity Mitigates Hallucination: By forcing the model to explicitly decide if a question is answerable (ac) and find the evidence (ss) before writing the response, the system is much less likely to "make things up." SCoT prompting increased agent faithfulness to grounding documents by up to 16.8%.
  • The "Assistant" Advantage: Using a separate, instruction-tuned model for the reasoning steps (ac and ss) was significantly more effective than relying on the generator model alone.
  • Performance Gains: When used to augment existing human-labeled data, the synthetic examples improved F1-scores by 10% to 14% absolute. In out-of-domain evaluations, improvements of up to 13.9% in F1-score were observed.
F1-HM performance across different SCoT configurations.

Conclusion

SCoT prompting demonstrates that a structured reasoning process can significantly enhance the quality and faithfulness of LLM-generated conversational data. The approach allowed relatively smaller, open-source models to produce data quality comparable to much larger systems. Future work will focus on extending SCoT to multi-document grounding and handling more ambiguous user queries.

Summary updated

Abstract

We introduce a structured chain-of-thought (SCoT) prompting approach to generating content-grounded multi-turn question-answer conversations with a pre-trained large language model (LLM). At the core of our proposal is a structured breakdown of the complex task into a number of states in a state machine, so that actions corresponding to various subtasks, e.g., content reading and utterance generation, can be executed in their own dedicated states. Each state leverages a unique set of resources, including prompts and (optionally) additional tools, to augment the generation process. Automatic evaluation shows that SCoT prompting with designated states for hallucination mitigation can increase agent faithfulness to grounding documents by up to 16.8%. When used as training data, our open-domain conversations synthesized from only 6 Wikipedia-based seed demonstrations train strong conversational QA agents. In out-of-domain evaluation, for example, we observe improvements of up to 13.9% in F1-score against ground truth over target domain gold data when the latter is augmented with our generated examples.
BibTeX
@inproceedings{sultan2024structured,
  author = {Md. Arafat Sultan and Jatin Ganhotra and Ramón Fernandez Astudillo},
  title = {{Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=i9NSMJNCL2},
}
Figure