TL;DR
This paper introduces a novel pipeline that leverages Large Language Models (LLMs) to extract and visualize causal claims from natural language text, creating "causal claim networks." This approach aims to provide a more scalable and cognitively plausible way to understand how people represent causal beliefs, moving beyond traditional experimental methods.
What Problem Does This Paper Address?
Understanding how humans form and update their beliefs is fundamental to cognitive science. Causal reasoning plays a critical role in this process, shaping everything from learning and categorization to memory and explanatory preferences. However, misinformation about causal relationships (e.g., vaccine-autism links) can have serious societal consequences. While NLP has made strides in text analysis, accurately identifying and representing psychological causality – what people believe causes what – has been a persistent challenge. Existing methods often struggle with the sheer diversity of linguistic expressions of causality and tend to be brittle, failing to generalize well. This paper addresses the need for NLP tools that can produce representations of causal beliefs that are not only accurate but also cognitively plausible and useful for scientific understanding and intervention.
Methods
The core of the paper is a three-step pipeline:
- Causal Claim Extraction: A RoBERTa-XL transformer model, fine-tuned on a causal language task using a BIO (Before-Inside-Outside) tagging scheme, is used to identify cause-effect pairs within sentences or paragraphs. This model is described as outperforming previous state-of-the-art models on this task.
- Causal Topic Clustering: The embeddings of the extracted causal entities are clustered. This process, extending a BERTopic approach, groups semantically similar causal claims, effectively proxying for "causal topics" within the corpus. This simplifies the network by mapping similar claims to common nodes.
- Causal Claim Network Construction: The extracted cause-effect tuples are treated as directed edges, and the identified entities as nodes. These are then aggregated across the entire corpus to form a network where nodes represent causal topics and edges represent documents containing causal claims linking those topics. The edge thickness indicates the number of documents linking two clusters.
The pipeline is presented as an interactive web tool, allowing users to upload data, select relevant columns, and visualize the resulting causal claim network. Users can also opt for pre-processing and entity clustering.
Datasets
The paper uses a dataset of 6000 tweets about the Covid-19 vaccine for their case study. This dataset was curated by subsetting a larger sample of tweets from before and after the release of Covid-19 vaccines. The authors provide access to this dataset on their tool interface.
Key Insights and Ideas
- LLMs as a Bridge to Cognitive Plausibility: The paper highlights how LLMs, with their rich semantic representations and sub-word tokenization, are uniquely positioned to overcome the historical brittleness of NLP in identifying causal language, especially when causality is latent.
- Causal Networks as Belief Proxies: The core idea is that aggregating causal claims from a corpus can approximate the complex, interlocking belief systems people hold about a topic. This offers a data-driven way to explore these systems.
- Interactive Visualization for Exploration: The pipeline's interactive visualization is crucial, allowing researchers to explore causal relationships, identify central topics, and understand the breadth of considerations people have about a subject.
- Topic Modeling for Semantic Grouping: Clustering causal entities into topics is key to managing the complexity of large networks and revealing overarching themes in causal beliefs.
Key Results
The case study on Covid-19 vaccine tweets resulted in:
- 408 extracted causal claims.
- 9 distinct causal clusters (topics). These clusters were identified as: Death, Oxford vaccines, Pandemic, Pfizer vaccine, Side-effects, Pfizer shot, Immunity and antibodies, Coronavirus, and Covid vaccine.
- Table 2 shows the distribution of cause and effect word spans per cluster, indicating varying roles (e.g., "Death" is predominantly an effect, while "Immunity" is predominantly a cause).
The RoBERTa-XL model used for claim extraction demonstrated higher performance compared to smaller BERT models and previously reported state-of-the-art implementations on the causal relation identification task (as shown in Table 1).
Limitations and Future Work
Limitations: * The paper notes that entity clustering works best when there's an abundance of causal claims about semantically distinct entities. If clustering doesn't produce clear results, users are advised to deselect it. * Large CSV files are automatically truncated to the first 10,000 rows.
Future Work: * Expand the visualization tool to allow for richer interactions, such as building subnetworks based on other data attributes (e.g., stance, sentiment). * Develop quantitative measures for comparing and diverging across networks. * Test the efficacy of the pipeline for studying human reasoning and belief change in various domains. * Use the generated causal claim networks to inform the design of surveys for measuring people's conceptual systems about complex issues.
Summary updated