Abstract
Sparse linear inverse problems are fundamental to signal processing, yet real-world multi-task data often exhibits complex clustering rather than simple joint sparsity. This paper introduces Clustered Sparse Bayesian Learning (C-SBL), a framework that simultaneously clusters related tasks and learns their sparse representations. By exploiting hidden correlations without prior group knowledge, C-SBL significantly outperforms traditional multi-task and Bayesian methods in both synthetic benchmarks and dynamic scene reconstruction.
Introduction
The challenge of recovering a sparse vector $x$ from measurements $y = \Phi x + \epsilon$ is central to compressive sensing. In many applications, multiple related tasks (multiple $y$ vectors) are available. Standard approaches typically assume either total independence or complete joint sparsity across tasks. However, real-world data often forms latent clusters—for instance, snapshots of different moving objects sharing distinct support sets. C-SBL addresses this "blind" multi-task learning problem by integrating task clustering and sparse estimation into a unified objective function, avoiding the complexity and local minima issues of traditional hierarchical Bayesian models.
Methodology
C-SBL utilizes a generative model where the precision of each task is a linear combination of $K$ diagonal basis functions: $$\Gamma_j^{-1} = \sum_k w_{j,k} \Lambda_k^{-1}$$ Here, $\Lambda_k$ represents the shared sparsity profile of a cluster, and $w_{j,k}$ are weights assigning task $j$ to cluster $k$.
The framework optimizes a cost function derived from Type-II Maximum Likelihood. To ensure tractability, Jensen’s Inequality is applied to linearize the log-determinant term. A penalty function $f(w) = \beta w \log w$ is introduced to encourage "hard" clustering, effectively forcing tasks to associate with a primary basis function. The resulting model is solved using a Majorization-Minimization (MM) scheme, providing iterative update rules for sparse coefficients, cluster bases, and assignment weights. Theoretically, the authors provide a "semi-Bayesian" justification, demonstrating that in the noiseless limit, the global minimum corresponds to the uniquely maximally sparse solution.
Experimental Evaluation and Results
The performance of C-SBL was evaluated using synthetic datasets with randomly generated dictionaries and sparse vectors partitioned into five clusters, as well as dynamic image reconstruction tasks involving $64 \times 64$ resolution scenes.
As shown in the figures above, C-SBL consistently achieves lower Mean Squared Error (MSE) than standard SBL, Multi-task SBL (MMV), and Dirichlet Process (DP) methods. Notably, at low SNR (e.g., 15dB), C-SBL's performance closely approaches that of an "Oracle" estimator which possesses prior knowledge of the true clusters. This "SNR boosting" effect occurs because clustering allows the algorithm to effectively average out noise across related tasks. In dynamic scene reconstruction, C-SBL successfully identified distinct scene clusters from raw measurements, producing sharper images and faster convergence than competing algorithms.
Discussion and Conclusion
C-SBL's strength lies in the "power of intersection": the support of a task is defined by the intersection of the supports of its assigned basis matrices, leading to cleaner recovery. Furthermore, the clustering mechanism helps the algorithm navigate away from local minima that often trap individual task solvers like standard SBL.
While highly effective, future work could focus on automating the selection of the clustering penalty $\beta$ and optimizing the algorithm for high-dimensional applications, as current complexity scales quadratically with the feature dimension. Nonetheless, C-SBL represents a significant step forward in robust, blind multi-task sparse learning.
Summary updated