Inference from Stationary Time Sequences via Learned Factor Graphs

Nir Shlezinger, Nariman Farsad, Yonina C. Eldar, Andrea J. Goldsmith

Introduction

In signal processing and communications, researchers often face a trade-off between mathematically elegant but brittle model-based algorithms and powerful but data-hungry deep learning models. A hybrid approach, "Learned Factor Graphs for Inference from Stationary Time Sequences," proposes a "middle way." By embedding neural networks within the classical factor graph framework, this method achieves robustness and data efficiency.

The Challenge of Inference

Inference involves recovering a hidden sequence from observed data. Traditional model-based methods like the Sum-Product (SP) algorithm require precise statistical models, which are often unavailable. Conversely, end-to-end Deep Neural Networks (DNNs) require vast labeled datasets and lack flexibility regarding sequence length. The goal is to exploit known problem structures while learning unknown observation physics from data.

Methodology: Learned Factor Graphs

The framework utilizes factor graphs to represent complex joint probability distributions as local functions. By assuming stationarity—where underlying rules remain constant over time—the authors train a compact DNN to represent a single function node rather than the entire task.

Integration of a single learned neural network as a function node within a factor graph framework.

Two implementation strategies are proposed for these nodes:

  1. Classification Networks: Estimating state probabilities given observations.
  2. Conditional Density Networks: Modeling the actual probability density functions (PDFs).
Architectures for learned function nodes: (a) classification-based and (b) conditional density-based approaches.

These learned nodes are integrated into the classical SP algorithm to compute marginal probabilities, allowing for modularity and sequence-length flexibility. This hybrid approach requires significantly less data to reach high accuracy because the factor graph provides the structural "skeleton" while the DNN provides the specific probabilities.

Experimental Evaluation

The framework was tested on two distinct datasets:

  • Sleep-EDF Expanded Database: Classifying five sleep stages (Awake, REM, N1-N3) from whole-night EEG recordings.
  • Synthetic Communication Channels: Symbol detection in simulated Gaussian and Poisson channels with memory, where channel physics are unknown or changing.

Key Results and Insights

  • Accuracy and Efficiency: In sleep stage detection, a learned factor graph using a 34-layer ResNet achieved 84.2% accuracy, outperforming state-of-the-art end-to-end models. For symbol detection, it required approximately 20x fewer multiplications per time instance than standard Recurrent Neural Networks (SBRNN).
  • Small Data Robustness: With only 1,000 training samples from the Sleep-EDF data, the hybrid approach maintained 81% accuracy, whereas a standard ResNet failed to converge (60% accuracy).
  • Robustness to Uncertainty: In communication simulations, the learned factor graph significantly outperformed traditional methods when the assumed channel model was even slightly inaccurate (CSI uncertainty).
  • Personalization: The compact nature of the networks allows for training on individual patient data, yielding higher accuracy for specific individuals than massive models trained on general populations.

Conclusion

Learned factor graphs bridge the gap between theory and data-driven practice. While currently limited to cycle-free Markovian structures, future work aims to extend this to "loopy" graphs and explore unsupervised training methods to make these systems truly autonomous in changing environments.

Summary updated

BibTeX
@article{shlezinger2022inference,
  author = {Nir Shlezinger and Nariman Farsad and Yonina C. Eldar and Andrea J. Goldsmith},
  title = {{Inference from Stationary Time Sequences via Learned Factor Graphs}},
  journal = {IEEE TSP},
  year = {2022},
  doi = {10.1109/tsp.2021.3139506},
}
Figure