Introduction
In the field of Free-viewpoint Television (FTV), Depth-Image-Based Rendering (DIBR) is the standard technique for generating virtual viewpoints. However, the warping and hole-filling processes inherent in DIBR often introduce localized artifacts such as blurring, ghosting, and jaggedness, particularly around object edges. Traditional quality metrics like PSNR and SSIM frequently fail to capture these distortions because they rely on global pixel-wise comparisons rather than structural integrity. This paper proposes a new metric, SEIO (Statistics of Edge Intensity and Orientation), specifically designed to evaluate DIBR-synthesized images by analyzing the statistical distribution of edges.
Methodology: The SEIO Metric
The SEIO metric is a full-reference quality assessment tool that quantifies how much the "edge signature" of a synthesized image deviates from a reference. The pipeline consists of four primary stages:
- Edge Detection and Gradient Calculation: Using a Canny operator, the system identifies edges and computes horizontal and vertical gradients to determine the intensity (magnitude) and orientation (angle) of every edge pixel.
- Statistical Profiling: Instead of a pixel-by-pixel comparison, the method generates histograms for edge intensity and orientation. This captures the aggregate statistical character of the image's structure.
- Distance Measurement: The metric calculates the average distance between the reference and synthesized histograms.
- Pooling: The final quality score ($Q$) is a weighted combination of the intensity score ($Q_I$) and orientation score ($Q_O$). Empirical testing established weights of 0.65 and 0.35, respectively, indicating that edge intensity is a more significant factor in human perception of quality.
Experimental Setup and Results
The authors evaluated SEIO using the IRCCyN/IVC DIBR image database, which includes 12 reference images and 84 synthesized images with human Mean Opinion Scores (MOS). Performance was measured using the Pearson Linear Correlation Coefficient (PLCC).
Performance Comparison
The results demonstrate that SEIO significantly outperforms both traditional and specialized DIBR metrics:
- Traditional Metrics: PSNR (0.5098) and SSIM (0.5644) showed poor correlation with human judgment.
- Existing DIBR Metrics: 3DSwIM (0.6584) and RMP-PSNR (0.6772) were more effective but still lagged behind.
- Proposed SEIO: Achieved a PLCC of 0.8370, showing a much tighter alignment with human subjective scores.
Key Insights
- Edges as Quality Proxies: By focusing on edge statistics and ignoring flat areas, the metric effectively isolates the most perceptually jarring DIBR distortions.
- Parameter Sensitivity: The metric's robustness depends on selecting optimal parameters for histogram binning and gradient calculation. The authors found a "sweet spot" at 36 bins for orientation and 25 for intensity.
Conclusion
The SEIO metric represents a significant advancement in DIBR quality assessment by prioritizing edge statistics over global pixel differences. While currently limited by the need for a full reference image, its superior performance suggests it could be a foundational component for future no-reference metrics or real-time broadcast monitoring systems. Future research will focus on expanding the dataset and optimizing the computational overhead of edge detection.
Summary updated