Discovering Time Differential Law Equations Containing Hidden State Variables and Chaotic Dynamics

Takashi Washio, Fuminori Adachi, Hiroshi MotodaIJCAI 2005

Introduction

Automated scientific discovery aims to derive the mathematical laws governing physical processes directly from observational data. While early systems were restricted to static equations or required extensive domain knowledge, real-world systems are often dynamic, involve hidden state variables, and exhibit chaotic behavior. This paper introduces SCALETRACK (SCALE-types and state TRACKing based discovery system), a novel framework designed to discover differential equations from passive observations by leveraging general mathematical constraints and advanced state-tracking techniques.

Methodology

The SCALETRACK system operates through a multi-stage pipeline that integrates dimension estimation with scale-type constraints to prune the search space of potential equations.

The SCALETRACK discovery pipeline, illustrating the flow from measurement data to discovered equations.
  1. Dimension Estimation: The system uses correlation dimension analysis on measurement time series to estimate the number of active variables, including those not directly observed.
  2. Scale-Type Constraints: By identifying whether data follows ratio or interval scales, the system applies the "Extended Product Theorem" to generate mathematically admissible candidate formulae. This "physics-lite" approach eliminates millions of invalid equations without requiring specific domain laws.
  3. Candidate Generation and Evaluation: Candidate equations are tested using a Sequential Importance Sampling/Resampling Monte Carlo (SIS/RMC) filter. This particle-filter-based approach is robust against high nonlinearity and allows for the simultaneous estimation of parameters and hidden states.
  4. Parameter Optimization: The system incorporates integer rounding for coefficients, reflecting the tendency of physical laws to have integer-based constants, which improves the plausibility of the discovered models.

Experimental Evaluation

The framework was evaluated using synthetic datasets mimicking complex physical scenarios:

  • Two-dimensional formulae: Four cases (RR, RRH, RI, RIH) were tested, varying by scale types (Ratio vs. Interval) and the presence of hidden variables (H).
  • Altered Rossler Chaos Equation: A three-variable chaotic system used to assess the reconstruction of non-repeating attractors.
  • Noise Robustness: Performance was measured against relative noise levels ($\sigma_w$) ranging from 0.1% to 5.0%.

Results and Discussion

SCALETRACK demonstrated significant capability in recovering hidden variables and modeling chaotic dynamics. For systems with fully observed variables, the system remained robust up to 2.0% relative noise. In cases with hidden variables, the system successfully identified the correct equations provided noise levels remained below 1%.

Reconstructed attractor of the Altered Rossler Chaos system, demonstrating the system's ability to capture complex chaotic dynamics.

In the chaotic Rossler system test, SCALETRACK successfully reconstructed the attractor's visual shape. Although some estimated coefficients deviated slightly from the ground truth (e.g., 0.33 vs. 0.36), the underlying dynamics were preserved. Furthermore, the implementation on a 10-PC cluster provided a 10x speedup, reducing computation time for complex searches from months to days.

Despite these successes, the system faces challenges with computational intensity as the number of variables increases. Additionally, its sensitivity to noise increases when attempting to resolve hidden states, and it requires data that sufficiently covers the system's state space to ensure accurate discovery.

Conclusion

SCALETRACK represents a significant step toward automated scientific discovery in "black box" scenarios. By combining scale-type constraints with robust state-tracking filters, it can deduce the laws of complex, chaotic, and partially observed systems from raw data, paving the way for automated modeling in fields where first principles are not yet fully understood.

Summary updated

BibTeX
@inproceedings{washio2005discovering,
  author = {Takashi Washio and Fuminori Adachi and Hiroshi Motoda},
  title = {{Discovering Time Differential Law Equations Containing Hidden State Variables and Chaotic Dynamics}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2005},
}
Figure