Dimensionality Collapse: Optimal Measurement Selection for Low-Error Infinite-Horizon Forecasting

Helmuth J. Naumer, Farzad KamalabadiAISTATS 2023

Introduction: The Sensor Selection Dilemma

Predicting the long-term behavior of complex dynamical systems—ranging from atmospheric patterns to financial markets—is a cornerstone of modern science. A critical bottleneck in this process is "experimental design": determining which sensors to deploy to maximize forecasting accuracy. Traditional methods often focus on minimizing current state estimation error. However, this paper argues that being "good at the present" does not guarantee success in the "infinite horizon."

The Core Problem: Transient vs. Persistent Information

In many physical systems governed by differential equations, high-dimensional trajectories eventually settle onto lower-dimensional manifolds, such as limit cycles or chaotic attractors. Information regarding dimensions that quickly decay (transient noise) is irrelevant for long-term forecasting. The challenge lies in identifying the specific linear measurements that capture the "non-vanishing" components of the system's flow.

Methodology: A Shortcut to Optimal Design

The authors propose a method to minimize the Cramer–Rao Lower Bound (CRLB) for infinite-horizon forecasting. While a Dynamic Programming (DP) approach provides a theoretical baseline, it is computationally infeasible for high-dimensional systems.

The core innovation is a dimensionality reduction technique that identifies the optimal measurement subspace by analyzing the right singular vectors of the system's limiting flow. This "shortcut" reduces the complexity to $O(M^2)$ per step, where $M$ is the system dimension, making it applicable to large-scale problems.

Commutative diagram illustrating the mapping between state space and measurement parameters.

This mathematical framework allows the system to transform measurements between the original high-dimensional state space and the reduced manifold where the long-term dynamics reside.

Key Insights: The Geometry of Information

The effectiveness of this approach stems from the realization that Fisher Information for distant-future states is naturally low-rank. By prioritizing measurements aligned with the persistent manifold, the forecaster avoids wasting its "information budget" on transient dynamics.

Visualization of dimensionality collapse in a dynamical system, showing vector fields (A, C), trajectory convergence to a limit cycle (B), and the resulting optimization surface (D).

As shown in the figure above, trajectories (B) rapidly collapse from the ambient space onto a stable limit cycle. The optimization surface (D) reveals how information density is concentrated around this manifold, guiding the selection of optimal sensors.

Results and Future Directions

Simulations on the Van der Pol oscillator, the Lorenz system, and the Heat Equation demonstrate that this method captures nearly all the improvement potential of the expensive DP baseline. Furthermore, the method remains robust when integrated with an Extended Kalman Filter (EKF), proving its utility in scenarios where the true state is partially hidden.

Future work aims to extend this framework to "Active Sensing," where measurement selection is coupled with control inputs to actively drive the system into more informative regions of the state space. Additionally, combining this method with data-driven system identification (like SINDy) could allow for simultaneous model learning and optimal sensor placement.

Summary updated

BibTeX
@inproceedings{naumer2023dimensionality,
  author = {Helmuth J. Naumer and Farzad Kamalabadi},
  title = {{Dimensionality Collapse: Optimal Measurement Selection for Low-Error Infinite-Horizon Forecasting}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2023},
}
Figure