Abstract
Screen content images (SCIs), which combine computer-generated graphics, text, and natural backgrounds, present unique challenges for traditional image processing. This paper proposes a segmentation framework that separates the background and foreground using a sparse-smooth decomposition technique. By modeling the background as a smooth component and the foreground as a sparse component, the algorithm achieves superior performance in text and graphic extraction. Evaluated on HEVC test sequences, the method demonstrates a significant improvement in recall (0.95) compared to existing hierarchical clustering methods.
Introduction
Standard image compression standards like JPEG2000 or HEVC are primarily designed for natural photographic content characterized by smooth transitions. Screen Content Images (SCIs), however, contain sharp edges and high-contrast UI elements. Applying standard transform-domain coding to these sharp discontinuities often leads to energy leakage across coefficients, resulting in high bit-rates or visible "ringing" artifacts. To optimize compression and analysis, it is essential to segment SCIs into a smooth background layer and a sharp foreground layer.
Methodology
The proposed approach utilizes a Sparse-Smooth Decomposition model. The core premise is that the background can be represented by a linear combination of low-frequency basis functions, while the foreground is "sparse," occupying a small fraction of the image but containing high-frequency information.
The Mathematical Model
Each image block is modeled as: $$\hat{F}(x, y) = \sum_{k=1}^{K} \alpha_k P_k(x, y) + S(x, y)$$ Where $P_k(x, y)$ represents the first 10 low-frequency 2D Discrete Cosine Transform (DCT) bases, and $S(x, y)$ is the sparse foreground component. The separation is achieved by solving a Basis Pursuit problem, minimizing the $L_1$-norm of both the coefficients ($\alpha$) and the sparse component ($s$) using the Alternating Direction Method of Multipliers (ADMM).
To ensure computational efficiency, the authors implement a tiered pipeline:
- Flat Block Check: Immediate categorization of uniform color blocks.
- Smooth Block Check: Least-squares fit for purely smooth blocks.
- Sparse Decomposition: Full ADMM optimization for complex blocks.
Experimental Results
The algorithm was tested on a dataset of 332 image blocks derived from HEVC test sequences. The proposed scheme was compared against Shape Primitive Extraction and Coding (SPEC) and Hierarchical k-means (DjVu).
Quantitative Performance
The quantitative results highlight a major breakthrough in recall:
| Algorithm | Precision | Recall |
|---|---|---|
| SPEC | 0.50 | 0.64 |
| Hierarchical Clustering (DjVu) | 0.64 | 0.69 |
| Proposed Scheme | 0.64 | 0.95 |
A recall of 0.95 indicates that the algorithm captures nearly all foreground details, which is critical for maintaining legibility in compressed screen content. Unlike color-based clustering, this smoothness-based approach successfully handles cases where foreground and background colors overlap by focusing on the nature of the signal rather than just color distance.
Conclusion
By leveraging the physical nature of image signals—smoothness for backgrounds and sparsity for foregrounds—this decomposition method provides a robust solution for SCI segmentation. While computationally more intensive than simple clustering, its high accuracy makes it a strong candidate for advanced video compression, text extraction, and potentially medical imaging applications.
Summary updated