TL;DR
This paper introduces DeepCausality, a novel AI-powered framework that leverages transformers and named entity recognition to perform causal inference on unstructured text data. By applying this to the LiverTox database, they successfully identified key causal factors for drug-induced liver injury (DILI) and demonstrated its utility in stratifying patients by severity, achieving high concordance with expert assessments.
What Problem Does This Paper Address?
Manually assessing causality from large volumes of free-text medical documents is a monumental task, often proving impractical due to its time-consuming and labor-intensive nature. While traditional methods like randomized controlled trials (RCTs) are the gold standard for establishing causality, they are expensive and time-consuming. Observational data, such as electronic health records (EHRs) and clinical reports, contain a wealth of information but are largely unstructured. Extracting meaningful causal relationships from this unstructured data is a significant challenge. Existing AI approaches, while advancing rapidly, still struggle with robust causal inference from free text, often limited to structured datasets or requiring extensive domain-specific training. This paper aims to bridge this gap by developing a generalizable AI framework for causal inference directly from free-text medical information.
Methods
The DeepCausality framework integrates several key components:
- Data Preprocessing: Free-text documents are split into sentences, and each sentence is assigned an "endpoint" label based on the causal question being investigated (e.g., positive for sentences describing DILI, negative otherwise).
- Language Model (LM) Development: A transformer-based language model is employed. Specifically, they fine-tuned BioBERT, a BERT model pre-trained on biomedical corpora, using the LiverTox dataset. This allows the model to learn the joint probability distribution of words within the text.
- Named Entity Recognition (NER): A domain-specific NER tool, BERN, is used to identify biomedical-related entities (drugs, genes, diseases) within the text. This step is crucial for focusing the causal inference on relevant terms and reducing false positives.
- Causal Inference (Do-calculus): The framework adapts Judea Pearl's Do-calculus to infer causality from the extracted named entities. This involves comparing the probability of an endpoint occurring when a specific entity is "present" (DO(ner)) versus when it is "absent" (NOT DO(ner)) using statistical tests (specifically, a one-tail z-test).
The novelty lies in the seamless integration of these components into a general framework applicable to free text, particularly the use of domain-specific NER to refine the Do-calculus application.
Datasets
- LiverTox database: This dataset, curated by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) and the National Library of Medicine (NLM), contains information on drug-induced liver injury (DILI). It includes 1,095 drug records with various sections like Introduction, Background, Hepatotoxicity, and Mechanism of Liver Injury. For this study, text from four sections was extracted.
- ACG Clinical Guideline for idiosyncratic DILI: This guideline was used as a reference to verify the identified causal terms and to build a knowledge-based causal tree.
Key Insights and Ideas
- Bridging the structured-unstructured divide: The framework demonstrates that AI, specifically transformers and NER, can effectively extract causal insights from unstructured text, a significant challenge in real-world data analysis.
- Domain-specific NER as a crucial filter: Using BERN to identify biomedical entities significantly reduces the search space for causal factors, improving efficiency and accuracy by filtering out irrelevant terms.
- Do-calculus adapted for text: The adaptation of Do-calculus to work with named entities extracted from text provides a principled way to move beyond mere correlation towards inferring causal relationships.
- Knowledge-based causal trees for stratification: The ability to build a causal tree from identified factors and use it for patient stratification highlights the practical utility of the framework for clinical applications.
- Expert knowledge remains vital: The study emphasizes that while AI can accelerate discovery, expert knowledge is still indispensable for validating findings and building robust causal models.
Key Results
- High prediction performance: The fine-tuned BioBERT model achieved an accuracy of 0.92 and an F1-score of 0.84 for iDILI sentence prediction on the test set.
- Accurate causal term identification: DeepCausality identified 20 causal factors for iDILI, with 18 of them (90%) being consistent with current clinical knowledge as defined by the ACG guideline.
- Strong concordance with experts: The iDILI severity scores generated by DeepCausality showed a high concordance (0.91) with those assigned by domain experts.
- Significant data compression: Biomedical NER reduced the number of unique words from 15,804 to 87, achieving a 99.4% compression rate.
- Robustness: The framework demonstrated highly reproducible results across multiple runs, with an average percentage of overlapped terms (POT) of 0.923.
Limitations and Future Work
- Domain Specificity: While the framework is generalizable, the current study focused on the biomedical domain (LiverTox). Further validation in other domains is recommended.
- Generative Transformers: The authors suggest exploring more advanced generative transformers (like GPT-3) that might require less intensive task-specific training for causal inference.
- Cofounders and Colliders: DeepCausality can identify causal factors but cannot distinguish between confounders and colliders. Future work could involve developing directional Do-calculus statistics within Bayesian networks derived from transformers to address this.
- Integration of Domain Knowledge: The study highlights the importance of expert knowledge in building the causal tree. Future work could explore integrating domain-specific ontologies or knowledge graphs directly into the framework.
Summary updated