Adaptive Parameters Adjustment for Group Reweighted Zero-Attracting LMS

Danqi Jin, Jie Chen, Cédric Richard, Jingdong ChenICASSP 2018 · DOI

TL;DR

This paper introduces adaptive parameter adjustment for Group Reweighted Zero-Attracting LMS (GRZA-LMS) algorithms to improve system identification in group-sparse scenarios. By minimizing the mean-square deviation at each iteration, they derive optimal step sizes and regularization parameters online, leading to faster convergence and lower estimation errors.

What Problem Does This Paper Address?

Adaptive filtering algorithms, particularly the Least Mean Squares (LMS) algorithm, are crucial for online system identification. While variants like Zero-Attracting LMS (ZA-LMS) and its reweighted form (RZA-LMS) excel at identifying sparse systems (where only a few parameters are non-zero), they face a trade-off between sparsity and estimation bias, as well as convergence speed and steady-state performance. This challenge is amplified when dealing with group sparsity, where non-zero parameters tend to cluster together. Existing algorithms for group-sparse systems, like GZA-LMS and GRZA-LMS, require careful manual tuning of their step size and regularization parameters to achieve optimal performance. This paper aims to automate this tuning process by developing adaptive parameter adjustment strategies for these group-sparse LMS algorithms.

Methods

The core contribution of this paper is the development of Variable-Parameter GZA-LMS (VP-GZA-LMS) and Variable-Parameter GRZA-LMS (VP-GRZA-LMS) algorithms. These algorithms adapt both the step size ($\mu$) and the regularization parameter ($\rho$ or $\lambda$) online. The adaptation is driven by minimizing the mean-square deviation (MSD) of the weight error vector at each time instant.

Key methodological steps include:

  • Modeling Transient Behavior: The authors analyze the transient behavior of the MSD of the weight error vector using a mathematical model. This involves making standard assumptions in adaptive filtering literature, such as the statistical independence of the weight error vector from the input regressor.
  • Optimization Problem Formulation: By minimizing the MSD at the next time step ($n+1$) with respect to the current step size and regularization parameter, a quadratic optimization problem is formulated.
  • Closed-Form Solution: Under certain assumptions (like a positive definite Hessian matrix), a closed-form solution for the optimal step size and regularization parameter is derived.
  • Approximation for Online Implementation: Since the exact statistics required for the closed-form solution are not available in an online setting, the authors introduce approximations for these quantities using instantaneous estimates and temporal smoothing.
  • Temporal Smoothing: To ensure stability and smooth parameter evolution, temporal smoothing is applied to the adapted step size and regularization parameters.

Datasets

Not discussed in the paper. The evaluation is performed using simulated data generated according to specific system models and input signal characteristics.

Key Insights and Ideas

  • Automated Parameter Tuning is Crucial: The paper strongly emphasizes that the performance of group-sparse LMS algorithms is highly sensitive to parameter settings. Automating this tuning process is key to unlocking their full potential.
  • MSD Minimization as an Online Objective: Using the mean-square deviation (MSD) as the objective function for online parameter adaptation provides a principled way to balance convergence speed and steady-state error.
  • Closed-Form Solutions Enable Real-Time Adaptation: Deriving closed-form solutions, even if requiring approximations for online use, is a significant step towards practical adaptive parameter control.
  • Robustness to Assumption Violations: The proposed algorithms demonstrate effectiveness even when assumptions about input signal statistics (white, Gaussian) are violated, highlighting their practical utility.
  • VP-GRZA-LMS Outperforms: The variable-parameter version of GRZA-LMS consistently shows superior performance, achieving lower steady-state MSD and faster convergence, especially in challenging scenarios.

Key Results

  • Improved Convergence and Steady-State Performance: The proposed VP-GZA-LMS and VP-GRZA-LMS algorithms consistently outperform their fixed-parameter counterparts (LMS, GZA-LMS, GRZA-LMS) and other variable step-size algorithms in terms of both convergence speed and steady-state Mean Squared Deviation (MSD).
  • VP-GRZA-LMS Leads: In experiments with both white Gaussian and non-Gaussian correlated inputs, VP-GRZA-LMS achieved the lowest steady-state MSD and fastest convergence among all tested algorithms.
  • Effective Parameter Adaptation: Figures 3(a) and 3(b) demonstrate that the algorithms adapt the step size and regularization parameter effectively. They tend to use larger values initially to promote tracking and sparsity, then gradually reduce them to minimize MSD.
  • Robustness: The algorithms maintain good performance even with non-Gaussian correlated inputs, where standard assumptions are relaxed.

Limitations and Future Work

  • Approximations: The online implementation relies on approximations for statistical quantities that are not directly available. While shown to be effective, these approximations might introduce some suboptimality.
  • Parameter Smoothing Factors: The choice of temporal smoothing factors ($\gamma$ and $\gamma'$) for parameter adaptation is not explicitly optimized and might require tuning.
  • Computational Complexity: While the derived solutions are closed-form, the online computation of these parameters adds some overhead compared to fixed-parameter algorithms. The paper does not extensively detail this complexity.
  • Future Work: The authors suggest exploring further improvements in parameter adaptation and potentially extending these adaptive strategies to other adaptive filtering algorithms. Investigating the theoretical bounds of performance under various conditions could also be a future direction.

Summary updated

BibTeX
@inproceedings{jin2018adaptive,
  author = {Danqi Jin and Jie Chen and Cédric Richard and Jingdong Chen},
  title = {{Adaptive Parameters Adjustment for Group Reweighted Zero-Attracting LMS}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2018},
  doi = {10.1109/icassp.2018.8462004},
}
Figure