Abstract
In image processing and computer vision, the choice of a cost function significantly influences the outcome of image reconstruction and decomposition tasks. While Mean Squared Error (MSE) is the standard metric for optimization, the Structural Similarity Index (SSIM) and Pearson’s Correlation Coefficient (PCC) are often preferred for their alignment with human visual perception. This paper provides a rigorous mathematical analysis of these three metrics within the framework of linear decomposition. The authors demonstrate that MSE, SSIM, and PCC are mathematically equivalent regarding the selection of basis vectors. Furthermore, they establish that the coefficients derived from these different metrics are related by a simple constant—the correlation coefficient itself—revealing a deep structural link between pixel-wise error and structural similarity.
Introduction
Measuring the fidelity of a reconstructed image is a fundamental challenge in image processing. Traditionally, Mean Squared Error (MSE) has been the dominant metric due to its mathematical simplicity and ease of optimization. However, MSE often fails to capture structural distortions that are obvious to the human eye. To address this, metrics like the Structural Similarity Index (SSIM) were developed to account for luminance, contrast, and structural information. Pearson’s Correlation Coefficient (PCC) serves as another statistical measure of similarity, focusing on the linear relationship between pixel sets.
Despite their conceptual differences, these metrics are frequently used interchangeably as cost functions in optimization problems, such as sparse coding or dictionary learning. This paper investigates whether switching from an MSE-based cost function to an SSIM-based one fundamentally alters the features (basis vectors) selected to represent an image.
Theoretical Framework
The study focuses on linear decomposition, where a target image vector $y$ is approximated by a linear combination of basis vectors $x_i$ from a dictionary, plus an offset $o$:
$$x = s_1x_1 + s_2x_2 + \dots + s_nx_n + o\mathbf{1}$$
Here, $x$ represents the approximation, $s_i$ are the coefficients (or weights), and $o$ is the DC component (offset). The authors analyze three distinct optimization schemes:
- MSE Minimization: Minimizing the squared Euclidean distance between $y$ and $x$.
- PCC Maximization: Maximizing the absolute value of the correlation coefficient $|r_{xy}|$.
- SSIM Maximization: Maximizing the structural similarity index $|SSIM(x, y)|$.
By deriving the partial derivatives for the coefficients $s_i$ and the offset $o$ for each metric, the authors compare the conditions required for optimality.
The Four Associations
The core contribution of the paper is the formalization of four mathematical associations that bridge these metrics:
- Equivalence of Optimization Goals: The authors prove that minimizing MSE, maximizing $|r_{xy}|$, and maximizing $|SSIM|$ are all mathematically equivalent to maximizing the absolute value of the correlation coefficient within their respective linear schemes.
- Identity of Basis Selection: In a greedy selection process (such as Orthogonal Matching Pursuit), all three metrics will select the exact same basis vectors from a dictionary to represent a target image. This implies that the "features" identified by the algorithm are independent of the choice between MSE and SSIM.
- Metric Convergence: The paper demonstrates that the act of minimizing MSE is functionally equivalent to maximizing $|SSIM|$ and $|r_{xy}|$ in the context of linear approximation.
- The Coefficient Ratio: While the selected bases are the same, the weights ($s_i$) assigned to them differ. The authors found a constant ratio between the MSE-based coefficients and the SSIM-based coefficients: $$\frac{s_i^{MSE}}{s_i^{SSIM}} = r_{xy}$$
Key Insights: Why SSIM Outperforms MSE
A significant finding of this work explains why SSIM-optimized reconstructions often appear superior to those optimized for MSE. In an MSE-based reconstruction, the variance of the approximation is limited to the covariance between the target and the reconstruction. This often results in a "muted" or blurred appearance because the variance of the output is lower than the variance of the source.
In contrast, SSIM-based optimization forces the variance of the approximation to match the variance of the target image exactly ($\sigma_x^2 = \sigma_y^2$). By preserving the original signal's variance, SSIM maintains the contrast and sharpness that MSE-based methods tend to suppress.
Limitations and Scope
This research is a purely mathematical proof and does not rely on empirical testing against specific datasets (e.g., ImageNet), as the derivations hold for any vector-based linear decomposition. However, several constraints apply:
- Non-Zero Variance: The proofs assume that image blocks have non-zero variance. In perfectly flat regions (constant pixel values), the denominators in SSIM and PCC formulas become zero, causing the associations to break down.
- Linearity: The findings are strictly limited to linear transformations. It remains to be seen if these associations persist in non-linear models, such as deep neural networks or non-linear manifold learning.
Conclusion
This paper provides a theoretical unification of MSE, SSIM, and PCC in the context of linear image decomposition. By proving that these metrics lead to the selection of identical basis vectors and establishing a clear mathematical relationship between their optimal coefficients, the authors offer a new perspective on image quality assessment. These insights not only clarify the relationship between pixel-wise error and structural similarity but also suggest potential applications in optimizing data storage and sparse representation algorithms.
Summary updated