Quantized generalized minimum error entropy for kernel recursive least squares adaptive filtering

Jiacheng He, Gang Wang, Kun Zhang, Shan Zhong, Bei Peng, Min LiarXiv 2023 · DOI · arXiv

Introduction

Adaptive filtering is a fundamental tool in time series prediction, with applications ranging from financial forecasting to medical signal processing. The Kernel Recursive Least Squares (KRLS) algorithm is widely used for its ability to model nonlinear systems. However, KRLS is optimized for Gaussian noise environments and often fails in real-world scenarios—such as underwater communications or EEG signal analysis—where noise is non-Gaussian.

While Information-Theoretic Learning (ITL) criteria like Minimum Error Entropy (MEE) offer robustness against such noise, they suffer from high computational complexity. Calculating the "Information Potential" typically requires quadratic complexity relative to the number of samples, making it impractical for high-frequency or long-term applications.

Methodology: Quantization for Efficiency

To address the complexity-robustness trade-off, this paper introduces a quantized version of the Generalized Minimum Error Entropy (GMEE) criterion. The proposed approach integrates quantization into the KRLS framework, resulting in two new algorithms: QKRGMEE (Quantized Kernel Recursive Generalized Minimum Error Entropy) and its simpler variant, QKRMEE.

Quantization as Compression

The core innovation lies in using a quantizer to "cluster" error samples. Instead of maintaining a full Parzen window of all past error values, the algorithm groups similar error values into a compact "codebook" of representative vectors ($H$). This significantly reduces the number of comparisons needed to estimate the error distribution.

The Kernel Trick

Like standard KRLS, these algorithms utilize the "kernel trick" to map input data into a high-dimensional feature space via a Mercer kernel (specifically the Gaussian kernel). This allows the algorithms to solve complex nonlinear problems as linear ones in the feature space while maintaining the robust properties of the GMEE criterion.

Experimental Evaluation

The performance of the proposed algorithms was validated using two benchmarks:

  1. Mackey-Glass (MG) Chaotic Time Series: A standard test for learning complex nonlinear dynamics.
  2. Real-world EEG Data: FP1 channel data, which is inherently noisy and non-Gaussian, providing a rigorous practical test.

Key Results

The experimental results demonstrate that quantization provides a massive reduction in computational load with minimal impact on accuracy. For instance, with a sliding window size ($L$) of 50, the quantized algorithms often reduced the number of active elements ($H$) to between 7 and 10.

MSE performance comparison across four noise scenarios (Rayleigh, Mixed-Gaussian, Gaussian, and Mixed-noise).

As shown in the figure above, QKRGMEE and QKRMEE achieve Mean Square Error (MSE) levels nearly identical to their non-quantized counterparts (KRGMEE/KRMEE). Crucially, they significantly outperform standard KRLS in non-Gaussian environments like Rayleigh and Mixed-Gaussian noise. The quantization threshold ($\gamma$) acts as a tunable "knob" for engineers, allowing them to directly balance computational speed against steady-state precision. Furthermore, formal performance analysis proves that the algorithm's mean error eventually converges, ensuring stability during long-term predictions.

Conclusion

By combining quantization with generalized entropy criteria, QKRGMEE and QKRMEE provide a robust and efficient solution for nonlinear adaptive filtering. These algorithms maintain the high performance of ITL-based methods while slashing the computational costs that previously limited their real-time application. Future research may focus on adaptive quantization thresholds to further optimize performance in dynamic noise environments.

Summary updated

BibTeX
@misc{he2023quantized,
  author = {Jiacheng He and Gang Wang and Kun Zhang and Shan Zhong and Bei Peng and Min Li},
  title = {{Quantized generalized minimum error entropy for kernel recursive least squares adaptive filtering}},
  year = {2023},
  eprint = {2307.01442},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.01442},
}
Figure