Learning Path Distributions Using Nonequilibrium Diffusion Networks

Paul Mineiro, Javier R. Movellan, Ruth J. WilliamsNeurIPS 1997

Abstract

This paper introduces diffusion networks, a class of recurrent neural networks (RNNs) governed by stochastic differential equations (SDEs) rather than deterministic ones. The authors derive a gradient formula that allows these networks to learn the entire probability distribution of continuous-time paths. This enables them to perform complex signal detection and filtering tasks with near-Bayesian optimality.

The Problem with "Average" Paths

Most traditional neural networks are designed for discrete-time sequences or deterministic continuous-time systems (ODEs). However, many real-world signals—such as the movement of an object, changing light levels, or sensor data—are continuous-time, continuous-valued stochastic processes.

A major limitation of standard RNNs is that they often converge on "average" paths. In many scenarios, the average of two valid paths is actually a failure. For example, if a robot needs to navigate around a tree, the average of a path going left and a path going right is a path that hits the tree directly.

Navigating around an obstacle

To solve this, we need models that can represent the distribution of possible paths, bridging the gap between neural networks and continuous-time statistical inference (like Kalman filters).

Diffusion Networks and Local Gradients

The authors propose Diffusion Networks, which are defined by an Itô SDE: $$dX(t) = \mu(t, X(t), \lambda)dt + \sigma dB(t)$$ * $X(t)$: The state of the network nodes. * $\mu$ (Drift): A deterministic function representing the network's "logic," parameterized by weights $\lambda$. * $\sigma dB(t)$: A Brownian motion term representing "synaptic noise."

The Core Innovation: The authors derive a formula for the gradient of the log-likelihood of an observable path. By using Girsanov’s Theorem, they transform the problem of finding a probability density in continuous space into a manageable calculation.

Crucially, they show that for certain activation functions (like quasi-linear sigmoidals), this gradient calculation becomes local in space and time. This means the network doesn't necessarily need "backpropagation through time" or complex time-unfolding to learn. The stochastic nature of the system allows for a learning rule that only requires information available at the current time and location in the network, making it potentially more biologically plausible and easier to implement in specialized hardware.

Empirical Performance: Sinewave Detection

The paper evaluates the model on a Noisy Sinusoidal Detection task. The training set consisted of 100 synthetic samples of a sine wave buried in additive Gaussian noise. The test set included 10,000 novel noisy stimuli used to generate a Receiver Operating Characteristic (ROC) curve, determining if a signal was present in the noise.

ROC Curve for Sinewave Detection

The diffusion network demonstrated remarkable ability to detect a signal in high noise:

  • Sensitivity ($d'$): The network achieved a $d'$ of 1.89.
  • Accuracy: This translated to an 82.7% correct detection rate for equal prior probabilities.
  • Comparison to Optimality: The theoretical "Ideal Observer" (the Bayesian limit) for this specific problem has a $d'$ of 2.0 (84.1% accuracy).

The diffusion network reached within 1.5% of the theoretical Bayesian limit using only 100 training examples and no explicit knowledge of the signal's structure.

Limitations and Future Work

While diffusion networks offer a powerful way to model multi-modal path distributions, there are remaining challenges:

  • Fixed Parameters: The current work does not address how to optimize the noise level ($\sigma$) or the initial state distribution ($\nu$); these were kept fixed during the experiments.
  • Computational Cost: Evaluating the gradient requires Monte Carlo sampling (simulating many paths), which can be computationally expensive compared to deterministic models.
  • Factorial Constraint: The authors highlight "factorial" architectures—where hidden units don't receive feedback from observable units—as a way to make Monte Carlo approximations efficient. However, more complex feedback loops between observable and hidden states remain a challenge for future optimization.

Summary updated

BibTeX
@inproceedings{mineiro1997learning,
  author = {Paul Mineiro and Javier R. Movellan and Ruth J. Williams},
  title = {{Learning Path Distributions Using Nonequilibrium Diffusion Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1997},
}

Loading related content...

Figure