Introduction
Gaussian Processes (GPs) are highly regarded non-parametric tools for regression due to their ability to provide principled uncertainty estimates. However, they possess a significant vulnerability: extreme sensitivity to outliers. Standard Gaussian Process Regression (GPR) assumes that noise follows a Normal distribution. Consequently, even a small number of outliers can disproportionately influence the regression curve, pulling it away from the true underlying signal.
This paper introduces a robust and scalable GPR framework designed to maintain accuracy even when up to 90% of the data consists of outliers. By integrating a mixture likelihood model with stochastic variational inference and sparse inducing variables, the authors present a solution that is both mathematically rigorous and computationally efficient enough for real-time applications.
Methodology
The proposed framework rests on three methodological pillars designed to address the dual challenges of robustness and scalability:
- Mixture Likelihood Model: Rather than simply adopting heavy-tailed distributions (like Student-t), the authors explicitly model data as a mixture. Inliers follow a standard Gaussian distribution, while outliers are assumed to be sampled from a uniform distribution. This ensures that outliers are treated as independent of the GP, preventing them from distorting the learned function.
- Variational Inference (VI): The authors derive a variational lower bound (ELBO) to jointly infer the status of each data point (inlier vs. outlier) and the model hyperparameters. This transforms a complex integration problem into a tractable optimization task.
- Scalability via Inducing Variables and SVI: To overcome the $O(n^3)$ complexity of standard GPs, the model utilizes "inducing variables" to summarize the data, reducing complexity to $O(nm^2)$. Integrating Stochastic Variational Inference (SVI) further allows the model to process data in mini-batches, enabling it to scale to massive datasets.
Experimental Evaluation
The model was rigorously tested across synthetic and real-world benchmarks:
- Synthetic Benchmarks: The 1D Neal and 10D Friedman datasets were used to evaluate accuracy under controlled outlier ratios reaching 80%.
- Computer Vision: Evaluated on YFCC100M, CPC, and HPatches for tasks like relative pose and homography estimation. These tasks involve thousands of points with high outlier ratios.
- Biological Imaging: The model was applied to spatial transcriptomics from mouse olfactory bulbs and human breast cancer biopsies for dense gene expression imputation.
Key Results and Applications
Superior Robustness and Precision
On the Neal dataset with 80% outliers, the proposed model achieved a Mean Absolute Error (MAE) of 0.03, significantly outperforming standard Variational Free Energy (VFE) methods which recorded an MAE of 0.76. In feature matching, the model reached a precision of 85.3% on the YFCC100M dataset, nearly doubling the performance of state-of-the-art deep learning methods like OANet (51.2%).
Real-Time Efficiency
A standout feature of the SVI implementation is its speed. The model can process 4,000 feature points in approximately 9.8 milliseconds. This efficiency makes it viable for high-speed computer vision pipelines where robust spatial consistency is required.
GP as a Motion Field
A creative insight from the paper is treating motion vectors between images as a GP. This allows the model to distinguish between spatially consistent "smooth" true matches and random, spurious outliers, effectively using the GP as a geometric filter.
Discussion and Future Work
While the uniform distribution for outliers proves highly effective for random noise, it may be a "weak assumption" in cases of structured, systematic sensor bias. Additionally, while the framework handles fixed hyperparameters well, optimizing kernel parameters in extremely high-dimensional spaces remains a challenge. Future research may explore deeper integration with neural networks (Deep GPs) to learn more complex feature representations while maintaining the robustness demonstrated here.
Summary updated