Introduction
As remote work, cloud gaming, and high-definition video conferencing become ubiquitous, evaluating the visual quality of Screen Content Images (SCIs) has become a critical task. However, SCIs—which include screenshots, remote desktop feeds, and web pages—are fundamentally different from the "natural images" (photos of the real world) that most Image Quality Assessment (IQA) metrics were designed to evaluate. While natural images are characterized by smooth gradients and continuous textures, SCIs are a hybrid of sharp text, icons, and pictorial elements.
The Challenge of Screen Content
Traditional metrics like SSIM or PSNR often struggle with SCIs because the statistical properties of sharp text edges are "unnatural" to them. Instead of building a complex new metric from scratch, researchers have proposed a clever alternative: "naturalizing" SCIs to make them look more like natural images to existing tools.
Methodology: The Naturalization Trick
The core of this approach is a pre-processing step using bicubic interpolation-based up-sampling.
Statistical Analysis with BAM
To quantify the difference between image types, the authors used a tool called the Block Activity Measure (BAM). Their analysis revealed a stark contrast: natural images typically exhibit a "left-side" distribution (where most blocks have low activity or smoothness), whereas textual regions in SCIs have high activity and a "right-side" distribution.
The Naturalization Process
By up-sampling an SCI using bicubic interpolation—which calculates new pixels based on a weighted average of 16 neighbors—the sharp, high-frequency edges of text and graphics are slightly "softened." Through empirical testing on the SIQAD database, the authors found that an up-sampling factor of 2.4 provided the optimal balance, shifting the SCI's statistical distribution to closely resemble that of a natural image without losing underlying quality information. This technique is essentially a form of domain adaptation, modifying the input data to fit the "expectations" of models trained on natural scenes.
Experimental Results
The study utilized the SIQAD (Screen Image Quality Assessment Database), which contains 20 reference SCIs and 980 distorted versions across seven types of common artifacts. The results showed dramatic improvements across both Full-Reference (FR) and No-Reference (NR) metrics:
- Full-Reference Metrics: For the popular IW-SSIM metric, the Pearson Linear Correlation Coefficient (PLCC) improved by 27.14%. MS-SSIM saw a gain of 17.72%.
- No-Reference Metrics: The gains were even more significant. The BRISQUE metric, which performed poorly on raw SCIs (PLCC of 0.0510), saw its performance jump to 0.3303—a 548% improvement.
- Robustness: The naturalization process made metrics much less sensitive to the "picture-to-text" ratio, leading to more consistent quality scores across different types of screen content.
Conclusion and Future Work
The discovery that a standard image processing technique like bicubic up-sampling can act as a bridge between two different image domains is a powerful insight. While effective, the 2.4x up-sampling increases the computational load for subsequent IQA metrics, which could be a bottleneck for real-time applications. Future research may explore more sophisticated naturalization techniques, potentially using machine learning to "translate" SCIs into the natural image domain even more effectively, or investigating how optimal scaling factors change with higher display resolutions like 4K or 8K.
Summary updated