Abstract
In high-dimensional data processing, recovering the underlying low-rank structure of a parameter matrix is essential for reducing computational complexity and improving estimation accuracy. However, traditional recovery methods often rely on the assumption of system stationarity or the Restricted Isometry Property (RIP), which are frequently violated in real-world applications such as adaptive control and wireless communications. This paper presents a novel two-stage online algorithm designed to identify the true rank and estimate parameters in non-stationary stochastic systems. By integrating Recursive Least Squares (RLS) with adaptive weighted nuclear norm regularization, the proposed method achieves "oracle properties," including finite-time rank identification and consistent, normally distributed parameter estimates. Experimental results across self-tuning regulators and MIMO channel estimation tasks demonstrate the algorithm's robustness and superior precision compared to conventional RLS and existing baselines.
Introduction
The concept of "low-rankness" is a fundamental principle in signal processing and control theory, suggesting that high-dimensional matrices often possess a much simpler, compressed latent structure. Identifying this structure allows for significant reductions in storage and computational overhead. Despite its importance, the majority of existing low-rank recovery techniques are designed for stationary environments where statistical properties remain constant over time.
In practical scenarios—ranging from adaptive control loops to time-varying communication channels—systems are inherently non-stationary. Furthermore, these systems often lack "persistent excitation," meaning the input signals may not be diverse enough to reveal the system's structure through standard means. This research addresses these gaps by introducing a mathematically rigorous, two-stage online algorithm capable of recovering low-rank matrices under unpredictable, non-stationary conditions without the need for reprocessing historical data.
Methodology
The proposed approach utilizes a two-stage online estimation framework that balances initial approximation with refined structural identification.
Stage 1: Adaptive Estimation
The algorithm begins by employing a standard Recursive Least Squares (RLS) estimator to generate an initial "rough draft" of the parameter matrix. This stage is crucial for handling the streaming nature of the data. Following the RLS update, a Singular Value Decomposition (SVD) is performed on the current estimate to prepare for structural refinement.
Stage 2: Weighted Regularization
The core innovation lies in the application of adaptive weighted nuclear norm regularization. Unlike standard nuclear norm minimization, which penalizes all singular values equally, this method utilizes adaptive weights derived from the Stage 1 estimates. These weights are specifically tuned to penalize smaller singular values more aggressively, effectively driving them toward zero to reveal the true rank of the matrix.
Crucially, this regularization step possesses an explicit, closed-form solution—a variation of soft singular value thresholding. This ensures that the algorithm remains computationally efficient and suitable for real-time, online updates in high-speed environments.
Experimental Evaluation
The algorithm was evaluated using two distinct simulation environments designed to mimic complex, real-world dynamics:
- Self-Tuning Regulator (STR): A classic closed-loop adaptive control system. In this setup, the data is inherently non-stationary because the control inputs are generated based on previous outputs, creating a feedback loop that shifts the data distribution over time.
- MIMO Channel Estimation: A Multiple-Input Multiple-Output (MIMO) wireless communication scenario. To test the algorithm's limits, the "pilot signals" used for channel calibration were derived from a 3MB non-stationary video file. This mimics the complex, time-varying nature of actual data transmission more accurately than synthetic stationary noise.
Key Insights and Results
The Oracle Property
A significant theoretical contribution of this work is the proof of the algorithm's "oracle properties." The algorithm is shown to identify the exact true rank of the parameter matrix within a finite number of observations. Once the rank is correctly identified, the parameter estimates converge to the true values as if the rank had been known a priori, achieving optimal statistical efficiency.
Performance Metrics
- Rank Identification: In simulations involving a 40x40 matrix with a true rank of 4, the proposed algorithm achieved perfect rank identification (zero error) across 20 independent trials. In contrast, standard RLS failed significantly, yielding a rank error of 36.
- Estimation Precision: The Parameter Estimation Error (ParaEstErr) for the proposed method was approximately $4.55 \times 10^{-4}$, nearly half the error produced by standard RLS ($9.36 \times 10^{-4}$).
- Communication Efficiency: In the MIMO channel estimation task, the algorithm significantly outperformed the P-SIGW (Polar-domain Simultaneous Gridless Weighted) baseline in terms of Normalized Mean Square Error (NMSE), particularly as the observation window increased.
Conclusion
This research provides a robust framework for online low-rank recovery in linear stochastic systems, even when those systems are non-stationary and lack persistent excitation. The use of adaptive weighting is key to bridging the gap between theoretical consistency and practical performance. While the current framework is optimized for linear systems, future work will explore extending these weighted regularization techniques to nonlinear dynamical systems, where the initial RLS stage may be replaced by more flexible nonlinear estimators.
Summary updated