Abstract
Parametric system identification methods estimate the parameters of explicitly defined physical systems from data. Yet, they remain constrained by the need to provide an explicit function space, typically through a predefined library of candidate functions chosen via available domain knowledge. In contrast, deep learning can demonstrably model systems of broad complexity with high fidelity, but black-box function approximation typically fails to yield explicit descriptive or disentangled representations revealing the structure of a system. We develop a novel identifiability theorem, leveraging causal representation learning, to uncover disentangled representations of system parameters without structural assumptions. We derive a graphical criterion specifying when system parameters can be uniquely disentangled from raw trajectory data, up to permutation and diffeomorphism. Crucially, our analysis demonstrates that global causal structures provide a lower bound on the disentanglement guarantees achievable when considering local state-dependent causal structures. We instantiate system parameter identification as a variational inference problem, leveraging a sparsity-regularised transformer to uncover state-dependent causal structures. We empirically validate our approach across four synthetic domains, demonstrating its ability to recover highly disentangled representations that baselines fail to recover. Corroborating our theoretical analysis, our results confirm that enforcing local causal structure is often necessary for full identifiability.
Introduction
In the field of system identification, the goal is to extract the underlying physical parameters—such as mass, friction, or spring constants—that govern a system's dynamics from observed data. Traditional approaches, such as Sparse Identification of Non-linear Dynamics (SINDy), are highly interpretable but require a predefined library of candidate functions. If the true governing equations are not representable within this library, these methods fail. Conversely, deep learning models can approximate complex dynamics with high accuracy but often result in "entangled" latent representations, where individual neurons do not correspond to distinct physical properties.
This paper addresses the fundamental problem of identifiability: providing mathematical guarantees that a neural network can uniquely recover the true physical parameters of a system from raw trajectory data without prior knowledge of the governing equations.
Theoretical Framework: Local Causal Identifiability
The authors bridge the gap between deep learning and system identification by framing the problem as one of Causal Representation Learning (CRL). They introduce a novel identifiability theorem based on the principle of sparsity.
The core insight is that physical parameters typically influence only a subset of a system's state components. For instance, a damping coefficient specifically affects velocity components. The researchers prove that if these causal influences are sparse, the parameters can be uniquely recovered (up to permutation and diffeomorphism).
Crucially, the paper extends this to local, state-dependent causal structures. In many systems, interactions are transient—such as the mass of a ball only influencing the system during a collision. The authors demonstrate that these local snapshots provide stronger disentanglement guarantees than a single global causal graph, as the intersection of different local graphs across a trajectory further constrains the possible parameter space.
The SPARTAN Architecture
To implement this theory, the authors developed SPARTAN (Sparsity-Regularized Transformer for Analytical Networks). SPARTAN is structured as a Variational Autoencoder (VAE) designed to discover these sparse causal relationships.
The architecture consists of:
- Parameter Encoder: Maps an input trajectory to a set of latent parameters $\hat{m}$.
- Sparse Causal Decoder: A transformer-based model that predicts the next state. It utilizes Sparse Attention and a "path loss" to enforce sparsity in the connections between latent parameters and state updates. This forces the model to find the simplest causal explanation for the observed dynamics.
Experimental Evaluation
The model was evaluated on four synthetic 2D environments with varying causal complexities:
- Dual Particle: Two masses with independent damping and a spring attachment.
- Local Particle: A system where a spring force only activates when the particle is far from the origin, testing state-dependent dynamics.
- Springs: Multiple particles connected by a set of potentially inactive springs.
- Bounce: Five balls colliding elastically, representing the most challenging task due to the discontinuous, collision-dependent causal structure.
Results and Discussion
The primary metric for success was the Mean-Max Correlation Coefficient (MCC), measuring the alignment between learned latents and ground-truth physical parameters.
SPARTAN achieved near-perfect disentanglement (MCC $\approx 1.0$) across all domains. In contrast, standard MLP and Transformer baselines failed to isolate individual parameters (MCC $\approx 0.4-0.6$). Furthermore, models restricted to global causal graphs (like VCD) failed in environments like "Bounce," where interactions are strictly local. This confirms the theoretical prediction that state-dependent sparsity is a critical requirement for identifiability in complex systems.
The authors also demonstrated the utility of these disentangled representations through interventions. By manually adjusting a learned latent variable, they could specifically alter the corresponding physical behavior (e.g., increasing "springiness") in the generated trajectories, proving that the model had truly isolated the underlying physics.
Conclusion
By combining causal representation learning with local sparse attention, this work provides a robust framework for interpretable system identification. While currently limited by the need to pre-specify the number of latent dimensions, the approach paves the way for discovering physical laws from high-dimensional real-world data and designing active learning strategies for robotic interaction.
Summary updated