Introduction
In video compression, the transform stage is critical for reducing spatial redundancy. By converting spatial pixel residuals—the difference between a predicted block and the actual block—into frequency coefficients, codecs can efficiently discard less important data. While the Discrete Cosine Transform (DCT-II) is the current industry standard, the Karhunen-Loève Transform (KLT) is theoretically the optimal linear transform due to its ability to perfectly decorrelate a signal.
Despite its theoretical advantages, traditional KLT has historically been impractical for real-world video codecs due to two primary issues: signal dependency and non-separability. Signal dependency typically requires "online training," where the encoder calculates a new transform matrix for every block, a process that is computationally prohibitive. Furthermore, traditional KLT treats an $N \times N$ block as a single $N^2$ vector, leading to massive matrix sizes (e.g., $256 \times 256$ for a $16 \times 16$ block) that demand excessive memory and processing power.
Proposed Method: Signal-Independent Separable KLT
To bridge the gap between KLT’s optimality and the practical constraints of the Versatile Video Coding (VVC) standard, researchers have proposed a Signal-Independent Separable KLT. This approach integrates into the standard coding pipeline as an alternative transform path.
The proposed framework relies on three core innovations:
- Separability: Instead of applying one massive matrix to the entire block, the transform is applied to rows and columns separately. For a $16 \times 16$ block, this reduces the matrix size from $256 \times 256$ to just $16 \times 16$, drastically lowering computational complexity.
- Offline Training: To eliminate the need for real-time calculation, transform matrices are trained offline using a massive dataset of residuals. This creates "universal" matrices that generalize across different video content.
- Implementation Efficiency: To prevent encoder-decoder drift caused by rounding errors, floating-point matrices are converted into fixed-point integers. The implementation also leverages SIMD (Single Instruction, Multiple Data) instructions to accelerate matrix multiplications.
Targeted Inter-Coding and Residual Selection
For Inter-coded (motion-compensated) blocks, KLT often produces results similar to DCT-II. To ensure KLT provides a meaningful benefit, the authors introduced a "Residual Selection" strategy. Instead of training on all residuals, they used a parameter ($\alpha$) to select only high-distortion residuals—those that DCT-II handles poorly. This teaches the KLT to act as a specialist for cases where the generalist DCT-II fails.
As shown in the experimental data, tuning the $\alpha$ parameter allows for the optimization of the BD-rate (Bjøntegaard Delta rate). Higher values of $\alpha$ focus the training on residuals that are most difficult to compress, leading to better overall coding efficiency.
Experimental Results
The proposed method was evaluated against the VVC reference software (VTM-1.1) using standard JVET test sequences not seen during the training phase.
- Bitrate Savings: The method achieved an average of 2.7% bitrate saving in "All Intra" (AI) mode and 1.5% in "Random Access" (RA) mode. Some 4K sequences reached peak savings of up to 3.9%.
- Complexity Trade-offs: While encoding time approximately doubled in AI mode due to the additional transform options the encoder must evaluate, the decoding time only increased by 1-4%. This minimal impact on decoding is crucial for consumer devices like smartphones and televisions.
Conclusion
By combining offline training with a separable matrix approach, this research demonstrates that a practical, signal-independent KLT can significantly improve video coding efficiency. While encoding complexity remains a challenge for real-time applications, the substantial bitrate savings and low decoding overhead make it a promising candidate for future standard enhancements. Future work will likely focus on fast-decision algorithms to reduce encoder search time and extending the technique to chroma components.
Summary updated