Analysis of Drifting Dynamics with Neural Network Hidden Markov Models

Jens Kohlmorgen, Klaus-Robert Müller, Klaus PawelzikNeurIPS 1997

Introduction

In the study of dynamical systems—ranging from industrial machinery to human physiology—we often encounter "nonstationarity," meaning the rules governing the system change over time. Traditional neural networks, such as multi-layer perceptrons, usually assume the underlying rules are static. When the system shifts, these models fail to represent the new dynamics or explain the structure of the change.

While previous research developed "switching" models—where a system jumps abruptly from one mode to another—many real-world processes are more fluid. There is often a "drift" period where the system is in a state of transition. This paper addresses the challenge of detecting, modeling, and quantifying these gradual drifts in an unsupervised way by combining an ensemble of neural network "experts" with a Hidden Markov Model (HMM) containing intermediate mixture states.

Methods: The Neural Network HMM

The authors propose a two-step framework for modeling nonstationary time series:

  1. Unsupervised Expert Training: The process begins with an ensemble of competing "prediction experts," specifically Radial Basis Function (RBF) networks. These experts are trained on a time series using a gradient-based method that encourages specialization. If a specific expert predicts the next data point well, its learning rate is boosted for that point. Over time, different networks specialize in different operating modes of the system.
  2. The Drift HMM: To handle transitions, the authors construct an HMM. Crucially, the states in this HMM aren't just the "pure" experts from the first step. They introduce mixture states, which represent a weighted linear superposition of two experts: $$f(\vec{x}_t) = a(t) f_i(\vec{x}_t) + (1 - a(t)) f_j(\vec{x}_t)$$ By defining a discrete set of these intermediate mixture levels (e.g., 32 steps between any two experts), the model can use the Viterbi algorithm to find the most likely path the system took through these states. This allows the model to distinguish between an abrupt switch and a smooth, continuous drift.

Datasets

The researchers evaluated their model on two distinct types of data:

  • Drifting Mackey-Glass Dynamics: A synthetic, high-dimensional chaotic system generated by a delay differential equation. They manually induced an exponential drift between two different delay parameters to see if the model could recover the "shape" of the transition.
  • Wake/Sleep EEG: Real-world physiological data from a single-channel EEG recording of a human during an afternoon nap. This was compared against a "gold standard" manual segmentation performed by a medical expert using multiple sensors.

Key Insights and Results

The Importance of the "In-Between"

The core insight of this work is that transitions themselves carry valuable information. By treating the transition as a mixture of known dynamics rather than a "noisy" period or a new discrete state, the mathematical shape of the drift can be extracted (e.g., identifying that a transition was exponential).

Automated Complexity Reduction

The method includes a way to prune redundant experts. By successively removing networks and checking the prediction error (RMSE), the authors demonstrated that a complex system could often be described by just two or three primary "pure" modes and the drifts between them.

Performance and complexity reduction on drifting dynamics

Synthetic and Physiological Validation

  • Synthetic Accuracy: On the Mackey-Glass data, the model perfectly reconstructed the exponential drift profile hidden in the data. It also correctly identified that only two predictors were necessary to describe the entire system, despite starting with six.
  • Sleep as a Drift: In the EEG analysis, the model found that "Sleep Stage I" (the transition between being awake and deep sleep) wasn't necessarily a unique state with its own rules. Instead, the HMM represented it as a linear mixture of "Wake" dynamics and "Sleep Stage II" dynamics.
  • Physiological Validation: The algorithm’s unsupervised segmentation showed remarkable accordance with the medical expert's manual assessment. It successfully identified sleep onset and "wake-up spikes" during the nap using only a single EEG channel, whereas the human expert required a full suite of polysomnography data. Furthermore, the HMM approach provided a much higher time resolution for state changes than manual medical scoring.

Limitations and Future Work

  • Fixed Transition Matrix: Because the number of mixture states can become massive (e.g., 896 states for 8 experts), the authors used a fixed transition matrix based on prior knowledge—assuming drifts are monotonic and infrequent—rather than training the matrix via the Baum-Welch algorithm. This limits the model's ability to learn totally unknown transition behaviors.
  • Computational Scaling: As the number of experts or the resolution of the drift increases, the state space grows quadratically, which could become a bottleneck for very complex systems.
  • Future Directions: The authors suggest applying this framework to larger physiological datasets to gain deeper insights into the "shape" of sleep disorders and other biological transitions that are currently poorly understood.

Summary updated

BibTeX
@inproceedings{kohlmorgen1997analysis,
  author = {Jens Kohlmorgen and Klaus-Robert Müller and Klaus Pawelzik},
  title = {{Analysis of Drifting Dynamics with Neural Network Hidden Markov Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1997},
}
Figure