Spectral Filtering for Linear Dynamical Systems

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 220 papers42 conceptsfrozen 2026-10-04

Learning and controlling Linear Dynamical Systems (LDS) has traditionally been a "hard" problem in machine learning. While linear systems appear simple, identifying their hidden states from noisy observations often leads to non-convex optimization landscapes, where traditional methods like Expectation-Maximization (EM) can get stuck in local minima or take exponential time to converge.

Researchers have recently bypassed these hurdles using spectral filtering. This approach replaces the need to explicitly reconstruct hidden states with a mathematically grounded "memory mechanism" derived from the spectral properties of Hankel matrices.

The Hankel Matrix as a Temporal Compressor

The core intuition behind spectral filtering is that the history of a stable dynamical system can be compressed without losing critical information. In a linear system, the relationship between past inputs and future outputs is governed by the system's impulse response. This relationship is captured by a Hankel matrix, a mathematical structure where each entry depends on the time-lagged interactions of the system.

As Elad Hazan, Karan Singh, and Cyril Zhang demonstrated in Learning Linear Dynamical Systems via Spectral Filtering, the eigenvalues of these Hankel matrices decay exponentially for stable systems. This means that even if a system has a theoretically infinite memory, its behavior can be reconstructed with high precision using only a small, polylogarithmic number of basis functions.

By convolving input sequences with the top eigenvectors of this matrix—a process called wave-filtering—we can lift raw observations into a compact representation that captures the system's long-term dependencies.

Turning Non-Convexity into Convexity

The breakthrough of spectral filtering is that it transforms system identification into a convex optimization task. Instead of trying to learn the specific $A, B, C, D$ matrices of a state-space model (which is non-convex), researchers like Holden Lee and Yi Zhang showed in Spectral Filtering for General Linear Dynamical Systems that one can learn a linear mapping from the filtered history to the output.

Because the filters (the eigenvectors) are fixed and derived from the Hankel structure, the only thing left to learn is the weight of each filter. This is a simple linear regression problem, which is globally solvable and computationally efficient. This method, often called the Spectral LDS Improper Predictor (SLIP), allows for logarithmic regret in unknown systems, as explored by Paria Rashidinejad and Stuart Russell in SLIP: Learning to predict in unknown dynamical systems with long-term memory.

Universal Control and Stability Margins

A remarkable property of these spectral filters is their universality. In control tasks, we often don't know the exact dynamics of a robot or a power grid. However, Anand Paresh Brahmbhatt, Gon Buzaglo, and Sofiia Druchyna have shown that spectral filters can be constructed based solely on a target stability margin—a bound on the system's spectral radius.

In Efficient Spectral Control of Partially Observed Linear Dynamical Systems, they introduce Double Spectral Control (DSC). This algorithm uses a two-stage filtering process to manage adversarial disturbances. Because the filters depend only on the stability margin rather than the specific system matrices, they can be precomputed and applied as "plug-and-play" components. This leads to an exponential speedup in runtime compared to previous methods that slowed down as systems approached the edge of instability.

From Control Theory to Sequence Modeling

The principles of spectral filtering are now being applied to modern deep learning through Spectral Transform Units (STU). Traditional State Space Models (SSMs) like Mamba learn their transition matrices, which can lead to optimization instabilities. In contrast, STUs use fixed convolutional filters derived from Hankel matrices to represent temporal dynamics.

As detailed by Yubo Liu and Evan Dogariu in Flash STU: Fast Spectral Transform Units, this approach allows the network to focus its learning capacity entirely on data transformation rather than the memory mechanism itself. This results in:

  1. Stable Training: Bypassing the non-convexity of learning recurrences.
  2. Length Generalization: Annie Marsden and Naman Agarwal proved in Provable Length Generalization in Sequence Prediction via Spectral Filtering that these models can effectively extrapolate from short training sequences to much longer test sequences.
  3. Efficiency: SpectraLDS provides a way to distill these complex convolutional models back into simple, recurrent LDS for constant-time-per-token inference.

By grounding sequence modeling in the spectral theory of linear systems, this research direction provides a bridge between the provable guarantees of control theory and the expressive power of modern neural architectures.

Papers in this version

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 9 papers
  2. Version 1 · 2026-10-04 · 20 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_fixed_spectral_filtering_bbf27417,
  title = {fixed spectral filtering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-spectral-filtering/rdr_bbf27417cc924535b17bffed47db0399},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_spectral_filtering_for_linear_dynamical__bbf27417_v1,
  title = {Spectral Filtering for Linear Dynamical Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spectral-filtering-for-linear-dynamical-systems/rdr_bbf27417cc924535b17bffed47db0399/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}