Abstract
Automated discovery of causal relationships from text is a challenging task. Current approaches, which are mainly based on the extraction of low-level relations among individual events, are limited by the shortage of publicly available labelled data. Consequently, models often perform poorly when applied to distributionally different domains. To overcome this, we leverage dependency trees and adversarial learning to address adaptive causality identification and localisation. We present the Adaptive Causality Encoder (ACE) and a new causality dataset, MedCaus, which integrates various causality types. Our experiments across four benchmark datasets demonstrate a performance improvement of up to 7% over existing baselines.
Introduction
Identifying "cause and effect" in natural language is a cornerstone of human reasoning, yet it remains a significant challenge for AI. Current models struggle with two major hurdles: the complexity of causal expressions (implicit, unmarked, or nested) and domain shift. A model trained on medical journals often fails when applied to financial news due to the scarcity of labelled data in the target domain. This research introduces a framework that understands the underlying structure of causality well enough to adapt to new domains without requiring new labels.
Methodology
The proposed framework consists of two interconnected architectures designed to capture both linguistic structure and domain-invariant features.
1. Graphical Causality Encoder (GCE)
To understand a sentence, the model requires more than just a sequence of words; it needs grammatical structure. The GCE processes the dependency tree of a sentence to capture how words relate to one another.
- Graph Convolutional Networks (GCNs): These process the dependency tree to extract structural features.
- BiLSTM: A sequence model that captures the order and context of words.
- Fusion: By concatenating structural (GCN) and contextual (BiLSTM) features, the model creates a representation used for both identification (detecting a causal relation) and localisation (identifying the specific cause and effect spans).
2. Adaptive Causality Encoder (ACE)
To handle domain shift, the authors implement adversarial learning. This involves a "domain discriminator" that attempts to identify whether a sentence originates from the source domain (e.g., Medical) or the target domain (e.g., Financial).
The core encoder is trained to "confuse" this discriminator. By doing so, it learns to extract features that are domain-invariant, focusing on the universal logic of causality rather than domain-specific jargon.
Datasets: Introducing MEDCAUS
The paper utilizes four key benchmarks, including a significant new contribution:
- MEDCAUS: A large-scale dataset of 15,000 medical sentences from Wikipedia and PubMed, manually annotated for explicit, implicit, and nested causality.
- FinCausal: Financial news data used for cross-domain testing.
- SemEval-10 Task 8: A classic dataset for semantic relations.
- BioCausal-Small: 2,000 biomedical sentences.
Key Results and Insights
- State-of-the-Art Performance: On the MEDCAUS dataset, the GCE model achieved an F1-score of 93.2% for identification, outperforming standard GCN and LSTM baselines.
- Superior Adaptation: In cross-domain scenarios (e.g., MEDCAUS → FinCausal), the ACE model improved the F1-score by up to 7% compared to non-adaptive methods.
- Structural Importance: Leveraging dependency trees via GCNs proved more effective than sequence-only models, as causal patterns often follow consistent grammatical structures even when vocabulary changes.
- Visual Alignment: t-SNE visualizations confirmed that the ACE model successfully aligned data distributions from different domains, making disparate features appear similar to the classifier.
Conclusion and Future Work
While the ACE framework represents a significant advancement in cross-domain causality discovery, it remains dependent on the accuracy of external dependency parsers. Future research may focus on expanding these techniques to cross-sentence causality and full-document causal mapping, moving beyond intra-sentence extraction to capture broader narrative logic.
Summary updated