Modeling complex, irregularly sampled time series has traditionally required high-capacity non-linear models like Recurrent Neural Networks (RNNs) or Gaussian Processes. However, a growing body of research suggests that we can achieve similar or superior performance using simple linear models by first mapping the data into a high-dimensional feature space known as the path signature.
This approach, rooted in rough path theory, effectively linearizes the relationship between a continuous path and the system it drives. By shifting the complexity from the model architecture to the feature extraction step, researchers have developed efficient, theoretically grounded methods for tasks ranging from survival analysis to the control of non-linear systems.
The Signature as a Universal Basis
At the heart of this direction is the signature transform. As explained by Andrey Kormilitzin and Ilya Chevyrev in A Primer on the Signature Method in Machine Learning, the signature of a path is an infinite collection of iterated integrals that capture its geometric essence.
The fundamental "Aha!" moment of this theory is that any continuous function of a path can be approximated by a linear combination of its signature components.
In Learning from the past, predicting the statistics for the future, learning an evolving system, Daniel Levin, Hao Ni, and Terry Lyons demonstrate that this property allows non-linear regression problems to be solved with high accuracy using simple linear models. The signature acts as a "universal basis" for paths, much like polynomials act as a basis for functions on a real interval.
Linearizing Controlled Differential Equations
The power of the signature transform becomes most apparent when modeling systems governed by Controlled Differential Equations (CDEs). A CDE describes how a hidden state $h_t$ evolves in response to an external driving path $X_t$:
$$dh_t = f(h_t) dX_t$$
While $f$ may be highly non-linear, Adeline Fermanian, Pierre Marion, and colleagues show in Framing RNN as a kernel method: A neural ODE approach that the solution to such an equation can be represented as a linear function of the input path's signature.
This transformation is analogous to a Taylor expansion for paths. Just as a Taylor series approximates a function near a point using its derivatives, the signature expansion approximates the flow of a differential equation using iterated integrals of the control path. This allows researchers to replace complex neural architectures with linear estimators that operate on the signature space.
Applications in Sparse and Irregular Data
Because the signature is defined for continuous paths, it is naturally robust to irregular sampling and missing data—common headaches in real-world time series.
Survival Analysis with CoxSig
In Dynamic Survival Analysis with Controlled Latent States, Linus Bleistein, Adeline Fermanian, and Agathe Guilloux introduce CoxSig. This framework models individual event risks (like patient mortality) as solutions to CDEs. By using the signature transform to linearize these dynamics, they create a parsimonious estimator that outperforms black-box deep learning models on high-dimensional, irregularly sampled medical data.
Interacting Systems with SigLasso
Similarly, in Learning the Dynamics of Sparsely Observed Interacting Systems, Linus Bleistein and co-authors propose SigLasso. This method uses the signature to transform the task of learning non-linear interactions between multiple time series into a high-dimensional linear regression problem. By applying Lasso regularization to the signature terms, the model can identify which specific path interactions (e.g., the "lead-lag" relationship between two variables) are driving the system.
Why it Works: Intuition
The signature transform works because it explicitly encodes the order and interaction of events within a time window.
Consider a 2D path $(X, Y)$. The first-order signature terms are simply the total increments of $X$ and $Y$. The second-order terms include the "area" captured between $X$ and $Y$ (the Lévy area). As Hao Ni demonstrates in A multi-dimensional stream and its signature representation, these terms can linearly recover essential statistics like covariances and moments.
By mapping the raw data into this structured feature space:
- Non-linearity is handled by the transform: The iterated integrals capture the non-linear "twists and turns" of the path.
- Temporal dependencies are flattened: The sequence is converted into a single static vector (the truncated signature), allowing the use of standard regression tools.
- Invariance is built-in: Signatures are naturally invariant to "time-warping" (reparameterization), meaning the model focuses on what happened and in what order, rather than the specific speed at which it occurred.
As Anna Scampicchio and Melanie Zeilinger note in Data-driven control of input-affine systems: the role of the signature transform, this linearization simplifies even the most opaque dynamical systems, making them accessible to the well-understood world of linear control and optimization.