Domain Adaptative Causality Encoder

Farhad Moghimifar, Gholamreza Haffari, Mahsa Baktashmotlagh

Introduction

Identifying causal relationships—the "why" and "how" behind events—is a fundamental component of human reasoning. For Artificial Intelligence, however, extracting these links from natural language remains a significant challenge. While models can be trained to recognize causality in specific contexts, they often struggle when applied to new domains. The Adaptive Causality Encoder (ACE) framework addresses this by learning to identify the underlying structural "skeleton" of causality, allowing it to generalize across diverse topics, from medical literature to financial news.

The Challenge of Cross-Domain Causality

Current Natural Language Processing (NLP) systems face two primary hurdles in causal reasoning:

  1. Domain Shift: A model trained on medical data (e.g., "Virus X causes Disease Y") typically fails when applied to finance (e.g., "Inflation led to a market dip"). The vocabulary and stylistic nuances differ so vastly that the model cannot recognize the shared causal logic.
  2. Structural Complexity: Causality is not always signaled by explicit keywords like "because." It can be implicit, nested (where one causal event triggers another), or spread across complex grammatical structures.

To overcome these, a model must move beyond surface-level word patterns and capture the invariant syntactic structures that define causal relationships regardless of the subject matter.

Methodology: The ACE Framework

The researchers propose a two-tier architecture designed for both robust feature extraction and domain invariance.

Graphical Causality Encoder (GCE)

The GCE serves as the core feature extractor. It utilizes a Bi-directional LSTM to capture sequential word information and a Graph Convolutional Network (GCN) to process the sentence's dependency tree. By analyzing the grammatical relationships between words—such as which noun acts as the subject for a specific verb—the model identifies structural patterns that are more reliable indicators of causality than simple word order.

Adaptive Causality Encoder (ACE)

To achieve domain adaptation, the framework incorporates adversarial training. A domain discriminator attempts to identify whether a sentence originates from the source domain (e.g., Medical) or the target domain (e.g., Financial). Simultaneously, the encoder is trained to "confuse" this discriminator. This process forces the encoder to discard domain-specific jargon and retain only the universal features representing causal structures.

The ACE framework combines a Graphical Causality Encoder (GCE) for structural feature extraction with an adversarial component for domain adaptation.

Experimental Setup and Datasets

The study evaluates the framework across four key benchmarks:

  • MEDCAUS: A new dataset of 15,000 medical sentences labeled for explicit, implicit, and nested causality.
  • FinCausal: Financial news data used for causality identification and localization.
  • SemEval-10 (Task 8): A standard dataset for semantic relation identification.
  • BioCausal-Small: A collection of 2,000 biomedical sentences from PubMed.

For the localization task—identifying the exact spans of the cause and the effect—the model employs the IOBES (Inside, Outside, Beginning, End, Singleton) tagging scheme to ensure high granularity.

Key Results and Insights

The ACE framework demonstrated significant improvements in both accuracy and generalizability:

  • State-of-the-Art Performance: The GCE model achieved an F1 score of 93.2 on the MEDCAUS dataset, outperforming standard baselines like C-GCN and Bi-LSTM.
  • Enhanced Transferability: In cross-domain scenarios (e.g., training on medical data and testing on financial data), the adaptive approach improved performance by up to 7% over non-adaptive models.
  • Visualizing Domain Invariance: t-SNE visualizations confirm the model's success. Before adaptation, features from different domains appear as distinct clusters. After applying ACE, these distributions merge, indicating that the model has successfully learned domain-invariant representations of causality.
t-SNE visualizations showing the alignment of source and target domain features across different transfer tasks (MEDCAUS to SemEval, BioCausal, and FinCausal).

Conclusion and Future Directions

The ACE framework proves that focusing on syntactic structure over domain-specific semantics is key to building generalizable causal reasoning systems. By "un-learning" domain flavor through adversarial training, the model identifies the universal "causal skeleton" of language.

Future work will look to address the remaining challenges of implicit causality, where no clear trigger words exist, and the complexity of nested causal spans. Additionally, replacing GloVe embeddings with modern transformer-based architectures like BERT or RoBERTa may further enhance the model's performance.

Summary updated

BibTeX
@inproceedings{moghimifar2020domain,
  author = {Farhad Moghimifar and Gholamreza Haffari and Mahsa Baktashmotlagh},
  title = {{Domain Adaptative Causality Encoder}},
  booktitle = {ALTA},
  year = {2020},
  url = {https://openreview.net/forum?id=7YFCaPIgve},
}
Figure