Transient Analysis of Diffusion Recursive Least Squares via Signed Error Algorithm for Cyclostationary Colored Inputs

Wei Gao, Yi Xu, Jie Chen, Cédric Richard2022 · DOI

TL;DR

This paper introduces a robust adaptive filtering algorithm, DRLS-SE, designed for distributed networks operating under challenging conditions like impulsive noise and time-varying systems with cyclostationary inputs. The authors provide a theoretical analysis of its transient behavior, demonstrating its improved performance and robustness compared to existing methods.

What Problem Does This Paper Address?

In many real-world applications, adaptive filtering algorithms are deployed in distributed networks where multiple nodes collaborate to estimate an unknown signal. Traditional algorithms like Recursive Least Squares (RLS) offer fast convergence but can be sensitive to non-ideal conditions. Specifically, this paper tackles two major challenges:

  1. Impulsive Noise: Real-world environments are often contaminated by impulsive noise (sudden, large spikes), which can severely degrade the performance of adaptive filters.
  2. Cyclostationary and Nonstationary Inputs: The characteristics of the input signals themselves can change over time (nonstationary) or exhibit periodic variations (cyclostationary). This makes it difficult for algorithms to maintain accurate estimates.

Existing distributed adaptive algorithms, while addressing some of these issues, often struggle to provide robust and theoretically sound analyses under the combined presence of impulsive noise and dynamic input signals. This paper aims to fill this gap by developing and analyzing a robust diffusion-based RLS algorithm.

Methods

The core of this work is the development and analysis of the Diffusion Recursive Least Squares via Signed Error (DRLS-SE) algorithm. Here's a breakdown of the key methodological aspects:

  • Signed Error (SE) Adaptation: To combat impulsive noise, the algorithm incorporates a "signed error" mechanism. Instead of using the full error signal, it uses its sign. This effectively down-weights the impact of large error spikes caused by impulsive noise, making the algorithm more robust.
  • Diffusion Adaptation: The algorithm operates in a distributed network where nodes share information with their neighbors. This "diffusion" strategy allows for collaborative estimation, leveraging the collective knowledge of the network. The paper specifically uses the "adapt-then-combine" (ATC) diffusion strategy.
  • Transient Theoretical Analysis: The paper goes beyond steady-state analysis to investigate the algorithm's performance during the initial learning phase (transient behavior). This is crucial for understanding how quickly the algorithm converges and how it behaves under dynamic conditions.
  • Mathematical Modeling: The authors employ advanced signal processing and statistical analysis techniques, including matrix inversion lemmas, expectation calculations, and approximations (like Price's theorem), to derive closed-form expressions for the mean and mean-square error of the DRLS-SE algorithm.
  • Modeling of Challenging Conditions: The analysis explicitly accounts for:
  • Nonstationary Systems: Modeled using a random walk process for the optimal weight vector.
  • Cyclostationary Colored Inputs: Modeled using periodically time-varying input variances.
  • Contaminated-Gaussian (CG) Noise: A model that captures both Gaussian noise and impulsive interference.

Datasets

Not discussed in the paper. The paper focuses on theoretical analysis and uses simulated data generated according to the described models.

Key Insights and Ideas

  • Signed Error for Robustness: The simple yet effective "signed error" adaptation is a key insight for making RLS-based diffusion algorithms resilient to impulsive noise. This is a significant improvement over standard RLS, which is highly susceptible to such disturbances.
  • Transient Analysis is Crucial: Understanding the transient behavior is vital, especially for nonstationary systems and cyclostationary inputs. The paper shows that the algorithm's initial learning phase is well-behaved and predictable, even under these complex conditions.
  • Theoretical Models Validate Empirical Performance: The derived analytical models for mean and mean-square error closely match the simulation results. This provides strong evidence for the accuracy of the theoretical framework and the algorithm's effectiveness.
  • Trade-off in Cyclostationarity: The analysis reveals that the convergence behavior is influenced by the period of cyclostationary inputs. Faster variations (smaller T) present a greater challenge, but DRLS-SE still demonstrates superior performance.

Key Results

  • Robustness to Impulsive Noise: Simulations demonstrate that the DRLS-SE algorithm significantly outperforms standard diffusion LMS (DLMS) and non-robust RLS-SE algorithms in the presence of impulsive noise, avoiding performance degradation and divergence.
  • Accurate Theoretical Predictions: The empirical transient Mean-Square Deviation (MSD) curves closely align with the theoretical predictions, validating the analytical models developed in the paper.
  • Convergence in the Mean: The paper theoretically proves that the DRLS-SE algorithm converges to an unbiased estimate in the mean sense, even under the challenging conditions.
  • Outperformance of Non-Cooperative Methods: The DRLS-SE algorithm shows a clear advantage over non-cooperative RLS-SE, highlighting the benefits of diffusion adaptation.

Limitations and Future Work

  • Approximations in Analysis: The paper acknowledges the use of approximations in the mean-square analysis (e.g., for handling higher-order moments). While validated by simulations, these approximations might introduce slight discrepancies, particularly in the initial transient phase.
  • Computational Complexity: While not explicitly detailed, diffusion algorithms inherently involve communication overhead. Further research could explore reducing this communication burden while maintaining performance.
  • Generalization to Other Noise Models: The analysis focuses on Contaminated-Gaussian (CG) noise. Extending the analysis to other types of impulsive noise models could broaden the applicability of the DRLS-SE algorithm.
  • Real-World Deployment: While simulations are promising, testing the algorithm on real-world datasets with actual impulsive noise and cyclostationary signals would be a valuable next step.

Summary updated

Abstract

In this paper, we perform the transient theoretical analysis of diffusion recursive least squares via signed error (DRLS-SE) algorithm over networks in the presence of impulsive noise. The obtained analytical models allow us to investigate the impacts of nonstationary system and cyclostationary colored inputs on the network transient convergence behavior. Simulations are provided to highlight the robustness of DRLS-SE algorithm against impulsive noise, and corroborate the correctness and accuracy of obtained theoretical findings.
BibTeX
@misc{gao2022transient,
  author = {Wei Gao and Yi Xu and Jie Chen and Cédric Richard},
  title = {{Transient Analysis of Diffusion Recursive Least Squares via Signed Error Algorithm for Cyclostationary Colored Inputs}},
  year = {2022},
  doi = {10.1109/mlsp55214.2022.9943369},
}
Figure