The Challenging Growth: Evaluating the Scalability of Causal Models
Lists
Only you see this
Abstract
One of the pillars of causality is the study of causal models and understanding under which hypotheses we can guarantee their ability to grasp causal information and to leverage it for making inferences. Real causal phenomena, however, may involve drastically different settings such as high dimensionality, causal insufficiency, and nonlinearities, which can be in stark contrast with the initial assumptions made by most models. Additionally, providing fair benchmarks under such conditions presents challenges due to the lack of realistic data where the true data generating process is known. Consequently, most analyses converge towards either small and synthetic toy examples or theoretical analyses, while empirical evidence is limited. In this work, we present in-depth experimental results on two large datasets modeling a real manufacturing scenario. We show the nontrivial behavior of a well-understood manufacturing process, simulated using a physics-based simulator built and validated by domain experts. We demonstrate the inadequacy of many state-of-the-art models and analyze the wide differences in their performance and tractability, both in terms of runtime and memory complexity. We observe that a wide range of causal models are computationally prohibitive for certain tasks, whereas others lack in expressiveness. We release all artefacts to serve as reference for future research on real world applications of causality, including a general web-page and a leader-board for benchmarking.
BibTeX
@misc{tagliapietra2025challenging,
author = {Nicholas Tagliapietra and Juergen Luettin and Lavdim Halilaj and Moritz Willig and Tim Pychynski and Kristian Kersting},
title = {{The Challenging Growth: Evaluating the Scalability of Causal Models}},
year = {2025},
url = {https://openreview.net/forum?id=HBf6HFnpmH},
howpublished = {OpenReview},
note = {Submitted to ICLR 2025},
}