A Hybrid Framework for Explainable Aviation Risk: Causal Chain Extraction from Safety Reports with Physics-Based Failure Quantification and Interventional Analysis

Yutian Pang, Yifang Tian, Jiacheng Xie, Xinyu Zhao, Jueming Hu, John-Paul Clarke, Hans-Arno JacobsenarXiv 2026 · arXiv

This paper introduces a hybrid framework that blends Large Language Models (LLMs) with physics-based modeling to turn thousands of aviation accident reports into an explainable causal knowledge graph. By combining text-mined causal chains with mechanistic failure models, it enables researchers to perform "what-if" interventional analyses and quantify risk based on real-world physical conditions rather than just historical frequencies.

What Problem Does This Paper Address?

Aviation authorities maintain massive archives of accident reports, but most computational analysis of this data is limited to "sorting" reports into fixed categories. These purely statistical approaches fail to answer the most critical question for safety analysts: How much would the risk of a severe outcome have fallen if a specific contributing factor had been absent?

Existing methods struggle with two main issues:

  1. Lack of Mechanism: Purely data-driven models (like Bayesian networks fitted to codes) treat risk as a simple frequency. They don't understand the underlying physics—like how temperature and humidity actually cause ice to form in a carburetor.
  2. Selection Bias: Accident archives only record "reportable events." This creates a "collider" effect where different causes appear to compete with each other in the data, making it difficult to perform sound causal inference without expert-built structures.

Methods

The authors propose a two-layer architecture joined by a fusion rule:

  • Data Layer: A local LLM (Qwen3.6-35B-A3B) processes 56,202 NTSB and FAA report narratives. It extracts temporally ordered "causal chains" (e.g., moist air $\rightarrow$ induction icing $\rightarrow$ engine power loss $\rightarrow$ forced landing) using a closed vocabulary of 57 factors.
  • Physics Layer: For failure modes with physical mechanisms (like carburetor icing or fuel exhaustion), the authors attach mechanistic models. These models map continuous physical drivers (like temperature, dewpoint, and power settings) to an occurrence probability.
  • Fusion & Calibration: A "leaky noisy-OR" model combines these layers. Crucially, the physics layer provides the conditional shape of the risk (how it changes with weather), while the corpus data is used to calibrate the absolute level of risk to match the observed accident population.
  • Interventional Analysis: The framework supports Pearl’s "do-operator" in two ways:
  • Graph-Level: Using backdoor adjustment on the learned causal network for categorical factors.
  • Physics-Level: Directly transforming physical drivers (e.g., simulating the effect of applying carburetor heat) to see how the downstream risk changes.
Fig. 1. The hybrid framework. The data layer converts 56,202 report narratives into ordered causal chains and aggregates them into a causation graph. The graph gives the leading factors L(F), the consequence conditionals P(Y | F), the corpus base rates bP(F) and the per-category networks Gc. The physics layer maps the leading factors that have physical drivers to continuous variables θ and evaluates an occurrence model πF at the stated operating point. The admissibility monitor rejects any chain whose transitions the guard margins γe rule out at the recorded state. Calibration ties the level of the physical model to the corpus marginal, and a leaky noisy-OR fitted to the corpus combines the contributing factors of a failure mode. Interventional queries act through backdoor adjustment on Gc or through an intervention do(m) that transforms the physical drivers.

Datasets

The study utilizes a massive corpus of 56,202 reports drawn from three public archives:

  • NTSB Aviation Databases: 43,604 reports.
  • FAA Accident and Incident Data System (AIDS): 12,585 reports.
  • NTSB Published Investigation Reports: 13 detailed full-length reports.

For evaluation, the authors used a sample of 2,000 reports where LLM-extracted nodes were compared against investigator-flagged "cause" and "factor" codes.

Key Insights and Ideas

  • Physics as a Stable Prior: While data-driven models can "drift" as reporting practices change over decades, physics is constant. The authors found that the physics-based model was more stable across different time windows than purely statistical models.
  • The "Ceiling" of Predictability: By comparing their physics model to a supervised logistic regression, the authors found that their model reached an AUC of 0.61, very close to the theoretical "density ratio ceiling" of 0.64 for the given weather features. This suggests the physics model captures nearly all the available signal in the recorded weather data.
  • Selection Bias (Colliders): The paper highlights that in an accident-only database, causes often have a "lift" below one. For example, "procedure not followed" might appear negatively correlated with engine failure because if a mechanical failure is present, it "explains away" the accident, making the investigator less likely to flag a procedural error.
  • Interpretability over Accuracy: The authors honestly note that the physics layer adds only a small margin (0.03 AUC) to predictive accuracy. Its true value lies in interpretability—allowing an analyst to trace a risk reduction back to a specific physical intervention like "apply carburetor heat."

Key Results

  • High Extraction Fidelity: The LLM-based causal chain extraction recovered 90.8% of investigator-flagged causes from the narratives.
  • Icing Model Performance: The carburetor icing model successfully separated 1,205 icing accidents from 38,565 other accidents using only temperature and dewpoint.
  • Counterfactual Analysis: Using abduction on four specific accidents, the model estimated a "probability of necessity" for heat omission between 0.29 and 0.50. This quantifies how likely it is that the accident would have been avoided had carburetor heat been used.
  • Leading Factors: The framework identified "Fuel exhaustion or starvation" as the dominant contributor to engine failure, with a fitted strength ($p_i$) of 0.96.
Physics occurrence surface for carburetor icing, showing how the model separates icing accidents (orange dots) from the general accident population.

Limitations and Future Work

  • The "Safe Flight" Gap: Because the corpus only contains accidents, the model cannot calculate absolute per-flight risk without external "exposure" data (e.g., total flight hours in specific weather conditions).
  • Human Factors: While physical failures have mechanistic models, human and organizational factors (like "inadequate maintenance") remain purely data-driven. The authors suggest that future work could integrate cognitive or organizational models.
  • Model Idealization: The physics models are necessarily simplified. For instance, the carburetor icing model uses fixed cooling values ($25^\circ\text{C}$ at cruise, $32^\circ\text{C}$ at descent) which may vary across different aircraft engines.
  • Investigator Bias: Part of the model's performance might reflect "investigator practice," as investigators often use the same published icing charts the authors used to build their physics model.

Summary updated

Abstract

Aviation authorities hold decades of accident reports whose narratives record how investigators reconstructed each accident. Computational work on these archives has mostly sorted reports into fixed taxonomies. In this work, we propose a hybrid framework in which a text-based corpus identifies the leading factors of a failure mode. Physics-based models then quantify how likely that failure is at a stated operating point. Ordered causal chains extracted from 56,202 American accident reports with a local large language model aggregate into a causation knowledge graph. On an evaluation sample of 2,000 reports, the chains recover 90.8% of investigator-flagged causes. Each physical factor whose drivers the record carries receives an occurrence model calibrated to the accident population. A leaky noisy-OR fitted to the corpus combines the factors into a failure mode probability that supports intervention. For engine power loss, the carburetor icing model separates 1,205 icing accidents from other accidents using temperature and dewpoint alone, reaching 0.61 in area under the receiver operating characteristic curve against a density ratio ceiling of 0.64. Routine weather hours give no valid control, because flying concentrates in fair weather. On later years the physics adds 0.03 to that area for occurrence and nothing for severity, so its value lies in interpretability. On four named accidents, abduction gives a probability of necessity of 0.29 to 0.50 for the heat omission, under monotonicity. The Fast Causal Inference algorithm does not license the largest backdoor adjustment on the learned networks.
BibTeX
@misc{pang2026hybrid,
  author = {Yutian Pang and Yifang Tian and Jiacheng Xie and Xinyu Zhao and Jueming Hu and John-Paul Clarke and Hans-Arno Jacobsen},
  title = {{A Hybrid Framework for Explainable Aviation Risk: Causal Chain Extraction from Safety Reports with Physics-Based Failure Quantification and Interventional Analysis}},
  year = {2026},
  eprint = {2610.35658},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.35658},
}
Figure