Learning a model of a robot's dynamics—such as mapping joint positions to required torques—must happen in real-time. Historically, robotics researchers have faced a difficult trade-off between speed and principled statistical modeling. This paper introduces Incremental Local Gaussian Regression (LGR), a method that combines the principled, "black-box" parameter tuning of Gaussian Processes with the high-speed, incremental learning capabilities of Locally Weighted Regression.
The Problem: Speed vs. Principle
Two dominant approaches have historically tackled real-time regression in robotics, each with significant drawbacks:
- Locally Weighted Regression (LWR): This method is very fast and can learn incrementally as the robot moves. However, it isn't a "proper" statistical model. It relies on many "fiddly" manual tuning parameters that vary by dataset, making it hard to deploy robustly in new environments.
- Gaussian Process Regression (GPR): GPR is a gold standard for probabilistic modeling because it tunes its own parameters automatically. However, standard GPR is computationally crushing (scaling at $O(N^3)$ complexity) and usually assumes a "global" metric, making it struggle with non-stationary data where the "rules" of the function change depending on the robot's workspace position.
The authors tackle the challenge of creating a regression algorithm that is as fast and locally adaptive as LWR, but as principled and self-tuning as GPR.
Methodology: A Top-Down Approach
The authors propose a "top-down" approach to localization. Instead of training independent local models and averaging them (the "bottom-up" LWR approach), they start with a global Gaussian model and use a localizing function basis. They define features that combine a localizing weight (like a Radial Basis Function) with a local model (like a linear function).
To keep the math from becoming a computational nightmare, they introduce latent variables ($f_m^n$) that represent "virtual targets" for each local model.
This probabilistic backfitting decouples the models during the inference step, allowing each local model to update its parameters independently. By using a variational bound to approximate the posterior, the resulting algorithm operates with $O(MK)$ complexity (where $M$ is the number of local models and $K$ is the number of parameters per model)—matching the efficiency of LWR.
The algorithm learns incrementally, growing by adding new local models when the robot enters an unexplored area (determined by a weight threshold $w_{gen}$) and pruning unnecessary models using Automatic Relevance Determination (ARD).
Key Insights
- Coupling vs. Independence: In LWR, models are trained independently, which is fast but prevents them from "coordinating." LGR's loss function couples the models during training, forcing them to collaborate to fit the global data, while the latent variable trick keeps the computation local.
- Non-Stationarity as a Feature: Because each local model in LGR can have its own distance metric (length scales), the algorithm naturally handles non-stationary functions. A global GP would try to find one "average" length scale for the whole map; LGR adapts to the local "wiggliness" of the data.
- Principled Hyperparameter Updates: Unlike LWR, where shrinking length scales can lead to "catastrophic" overfitting, LGR’s updates are derived from maximizing the marginal likelihood. The model only gets more local if it actually improves the fit against the virtual targets.
Experimental Results
The method was evaluated on several inverse dynamics tasks of increasing complexity, including the standard Sarcos benchmark, real-world data from a KUKA lightweight arm (KUKA1 & KUKA2), and a large-scale simulated dataset (KUKAsim) containing 2 million data points.
LGR consistently achieved performance on par with or superior to state-of-the-art methods like LWPR and I-SSGPR (Incremental Sparse Spectrum Gaussian Process Regression). Notably, LGR achieved similar accuracy to LWPR but used significantly fewer local models.
Beyond model efficiency, the C++ implementation of LGR processed data at approximately 500Hz on a standard Intel Core i7, making it well-suited for high-speed robotic control loops. Furthermore, the authors demonstrated that LGR works across different datasets with the same initial settings, whereas LWR-based methods often require extensive cross-validation to find stable parameters.
Limitations and Future Work
While highly effective, LGR does have limitations. The authors note that the number of local models ($M$) can grow quickly in the early stages of learning before the ARD pruning mechanism has enough data to effectively remove redundant models. Additionally, while LGR reduces the number of tuning parameters, it still relies on a threshold ($w_{gen}$) to decide when to add a new model.
Future work may extend this "top-down" localization framework to other types of regression beyond linear local models, potentially incorporating more complex kernels within the local regions to further enhance predictive power.
Summary updated