Abstract
This paper introduces the Smoothing Machine (SMACH), a novel framework for performing "smoothing" in dynamical systems—the process of estimating hidden states using both past and future observations. By reframing a traditionally difficult unsupervised problem as a series of supervised learning tasks, SMACH provides a method that is more robust, easier to optimize, and theoretically sound compared to classical approaches like Expectation-Maximization (EM).
Introduction
In many fields, from robotics to speech recognition, inferring the "true" state of a system from noisy observations is a fundamental challenge. Smoothing is the gold standard for this task because it utilizes the entire sequence—both past and future context—to make the most accurate estimate.
Traditionally, this is handled by Latent Chain Conditional Random Fields (CRFs). However, these models are notoriously difficult to train because they involve latent variables, leading to non-convex optimization landscapes where algorithms like EM often get stuck in local optima. Furthermore, classical filters like the Kalman Smoother require precise mathematical models of a system's physics, which are often unavailable or inaccurate in real-world scenarios.
Methods
The core innovation of SMACH is moving away from modeling hidden states directly. Instead, it utilizes Predictive State Representations (PSRs), which track expectations of future and past observable features. The framework is built on three main components:
- Inference Machines (PSIMs): SMACH learns two stationary filters—one that passes "messages" forward through time and another that passes them backward.
- Supervised Learning & DAgger: Because predictive states are based on observables, training becomes a supervised regression or classification problem. The authors use the DAgger (Dataset Aggregation) algorithm to ensure the learners remain robust to their own errors over long sequences.
- The Combiner: A final model takes the forward message, the backward message, and the current observation to predict the final latent label.
By using "black-box" predictors like Kernel Ridge Regression, SMACH can capture complex, non-linear dynamics without needing to explicitly linearize the system.
Experimental Evaluation
The authors evaluated SMACH across robotics control and sequential classification tasks:
- Robotics Control: Tasks included Cart-Pole Balancing, Bicycle Balancing, Helicopter Hover, and Swimmer. SMACH consistently outperformed the Iterated Extended Kalman Smoother (iEKS), even when the iEKS was provided with the true underlying physics equations.
- Sequential Classification: On an Optical Character Recognition (OCR) dataset, SMACH-2 achieved a prediction error of 16.21%, vastly outperforming standard CRFs (32.30%). It also showed competitive performance on Natural Language Processing (NLP) tasks like the FAQ dataset.
Key Insights and Results
- The Power of k-Observability: The paper formalizes "k-observability," proving that a window of $k$ future/past observations can uniquely determine the latent belief state. As shown in the results, increasing the "look-ahead" window ($k$) leads to significantly better smoothing performance.
- Bypassing Model Mismatch: Unlike traditional filters that fail when physical models are slightly off, SMACH learns directly from data, making it resilient to "model mismatch."
- Global Optimality: By reducing sub-tasks to supervised learning with often convex objectives, SMACH avoids the local optima that plague EM-based latent variable models.
Limitations and Future Work
Despite its strengths, SMACH requires ground-truth labels for states during training, meaning it cannot currently operate in purely unsupervised environments. Additionally, its performance is sensitive to the choice of feature functions for predictive states. Future work may involve applying SMACH to more complex NLP tasks, such as Part-of-Speech tagging, where bidirectional context is critical.
Summary updated