Can ChatGPT Understand Causal Language in Science Claims?

Yuheun Kim, Lu Guo, Bei Yu, Yingya LiACL 2023

TL;DR

This paper investigates ChatGPT's ability to understand causal language in scientific claims, finding it lags behind specialized models and struggles with nuanced conditional claims. While prompt engineering, particularly Chain-of-Thought, can improve performance, finding optimal prompts remains challenging, highlighting ChatGPT's limitations in this domain.

What Problem Does This Paper Address?

Accurately describing causal relationships in scientific research is crucial but challenging. Authors often struggle with choosing language that precisely reflects the strength of evidence, leading to potential exaggeration or misinterpretation. Human readers also find it difficult to discern the strength of causal claims, especially when hedges are used. This paper addresses the question of whether large language models like ChatGPT can reliably understand and classify these causal claims, a task previously tackled by specialized NLP models.

Methods

The researchers evaluated ChatGPT (GPT-3.5) and GPT-3.5 using various prompt engineering strategies. They employed a zero-shot prompting approach, meaning the models were given instructions without explicit examples. Key prompt design elements explored included:

  • Baseline (BASE): A direct question asking if a sentence describes a causal or correlational finding.
  • Conditional/Possible Causal: Modifications to the baseline prompt to specifically address conditional causal claims, exploring whether "possible" was a better term than "conditional" for ChatGPT.
  • Chain-of-Thought (CoT): Adding instructions like "Let's think step by step" to encourage a reasoning process.
  • Context: Providing definitions and examples of causal, correlational, and no-relationship statements.
  • System Messages: Using predefined instructions to guide ChatGPT's behavior.

Performance was evaluated using a semi-automatic approach with heuristic rules to map model outputs to labels. Macro F1-score and Cohen's Kappa were used to measure accuracy and agreement.

Datasets

The study utilized two existing, manually annotated datasets of science claim strength:

  • PubMed dataset: 3,061 research conclusion sentences from structured abstracts.
  • EurekAlert! dataset: 2,076 sentences from health-related press releases.

Both datasets were annotated with four categories: correlational, direct causal, conditional causal, and no relationship. A subset of the PubMed dataset was used for development and prompt selection.

Key Insights and Ideas

  • "Conditional Causal" is a Misnomer for LLMs: ChatGPT struggled significantly with the "conditional causal" category, often misinterpreting it. Replacing "conditional" with "possible" in the prompt dramatically improved performance, suggesting the original term was confusing for the model.
  • Hedges are a Challenge: ChatGPT exhibited difficulty interpreting hedges (e.g., "may," "appear," "might") commonly used in conditional causal claims, mirroring confusion observed in human readers.
  • Prompt Engineering Matters, But It's Tricky: While Chain-of-Thought (CoT) and context-based prompts improved ChatGPT's performance, the gains were not always consistent. Adding too many elements or specific phrasing could even degrade performance.
  • LLMs Still Lag Behind Specialized Models: Even with optimized prompts, ChatGPT's performance on causal claim classification was substantially lower than fine-tuned BERT models previously developed for this task.
  • Inconsistency is a Concern: ChatGPT's performance varied across semantically similar prompts and even across different days, raising questions about its reliability as an off-the-shelf solution.

Key Results

  • Baseline Performance: The initial BASE prompt yielded a macro F1-score of .486, significantly lower than the .881 achieved by a fine-tuned BioBERT model.
  • Conditional Causal Improvement: Switching "conditional causal" to "possible direct causal" in the prompt boosted the F1-score for this category from .164 to .578, and the overall macro F1 from .486 to .631.
  • Best ChatGPT Prompt: The "BASE + CoT" prompt achieved the highest macro F1-score of .772 for ChatGPT on the development set.
  • Comparison with Fine-tuned Models: Even the best ChatGPT performance (.772 macro F1) fell short of the .881 macro F1 from a fine-tuned BioBERT model.
  • Dataset Performance: On the full PubMed and EurekAlert! datasets, macro F1-scores ranged from .628 to .698.
  • Ensemble Models: Ensemble methods using majority voting did not improve performance beyond the best individual prompts.

Limitations and Future Work

The authors acknowledge several limitations:

  • Development Set Size: The development set was relatively small (200 sentences) due to the time cost of post-processing ambiguous answers.
  • Zero-Shot Focus: The study primarily focused on zero-shot settings, and further exploration of few-shot prompting could be beneficial.
  • Prompt Engineering Methodology: A systematic method for inferring causality between prompt wording and model performance is still lacking.
  • Inconsistent Results: The variability in ChatGPT's performance across runs and prompts suggests caution when generalizing findings.

Future work could involve exploring few-shot prompting, developing more systematic methods for prompt design, and investigating how ChatGPT's understanding of causal language might be leveraged to improve human annotation guidelines.

Summary updated

BibTeX
@inproceedings{kim2023can,
  author = {Yuheun Kim and Lu Guo and Bei Yu and Yingya Li},
  title = {{Can ChatGPT Understand Causal Language in Science Claims?}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure