QoS to QoE Mapping Function for Iptv Quality Assessement Based on Kohonen Map: A Pilot Study

Jaroslav Frnda, Marek Ďurica, Mihail Savrasovs, Philippe Fournier‐Viger, Jerry Chun‐Wei LinTransport 2020 · DOI

Abstract

This paper deals with an analysis of Kohonen map usage possibility for real-time evaluation of end-user video quality perception. The Quality of Service framework (QoS) describes how the network impairments (network utilization or packet loss) influence the picture quality, but it does not reflect precisely on customer subjective perceived quality of received video stream. There are several objective video assessment metrics based on mathematical models trying to simulate human visual system but each of them has its own evaluation scale. This causes a serious problem for service providers to identify a critical point when intervention into the network behaviour is needed. On the other hand, subjective tests (Quality of Experience concept) are time-consuming and costly and of course, cannot be performed in real-time. Therefore, we proposed a mapping function able to predict subjective end-user quality perception based on the situation in a network, video stream features and results obtained from the objective video assessment method.

Introduction

In the world of IPTV, there is a persistent "perception gap" between technical network metrics and actual user satisfaction. Network administrators typically monitor Quality of Service (QoS)—metrics like packet loss, jitter, and bitrate. However, these numbers do not always tell the full story of Quality of Experience (QoE), which is how a human actually perceives the video. For instance, a 1% packet loss might be invisible in a static news broadcast but devastating in a high-motion sports game.

While subjective testing (asking humans to rate videos) provides the most accurate QoE data, it is too slow and expensive for real-time monitoring. Objective mathematical models like the Structural Similarity Index (SSIM) exist, but they often use different scales and do not perfectly correlate with human emotion or intervention thresholds. This pilot study aims to create a mapping function that can automatically translate technical network conditions and video features into a human-centric quality score in real-time using Self-Organizing Maps (SOM).

Methodology

The researchers employed a Kohonen Map, also known as a Self-Organizing Map (SOM). This is a type of unsupervised artificial neural network that excels at clustering high-dimensional data into a simple two-dimensional grid. Unlike standard backpropagation networks, SOMs do not require a target output vector during the initial training phase, making them highly robust against overfitting.

Key components of the methodology included:

  • Objective Metric: SSIM was used as a baseline objective measure because of its strong correlation with human vision.
  • Input Features: The SOM was trained on four key parameters: Codec type (H.264 vs. H.265), Bitrate (5, 10, 15 Mbps), Packet loss (0.1% to 1%), and Scene type.
  • Clustering: Instead of predicting a precise decimal Mean Opinion Score (MOS), results were grouped into three actionable "traffic light" clusters: Green (MOS ≥ 4), Orange (MOS 3–3.99), and Red (MOS < 3).
The SSIM objective metric framework used as a baseline for the SOM training.

Datasets and Experimental Setup

The study utilized a custom-created distorted video database consisting of 432 video sequences affected by varying levels of packet loss. To calibrate the model, 60 human observers (aged 18–35) provided subjective ratings using the Absolute Category Rating (ACR) method to establish a ground-truth MOS.

The dataset included four distinct scene types to test the model's versatility: night scenes, static scenes, sports, and "very dynamic" scenes with fast camera movement. The data was divided into a 95:5 ratio for training versus hold-out testing.

Examples of video content used in the study, showing different scene complexities from construction sites to high-motion marathon scenes.

Results and Analysis

The study tested various SOM topologies to find the optimal configuration for quality classification. A 10x10 grid was found to be the most effective, achieving a success rate of 86.4% in correctly classifying video quality into the three defined clusters. For the binary classification of "Good" vs "Bad" quality (using MOS 3 as the boundary), the model achieved a precision of 92.3% and a recall of 85.7%.

Key insights from the analysis include:

  • Codec Resilience: The SOM effectively captured non-linear relationships, such as H.265's increased sensitivity to data loss compared to H.264 due to its higher compression ratio.
  • Dimensionality Reduction: The SOM successfully mapped a complex 4D input space (bitrate, loss, codec, scene) onto a 2D grid where distinct "quality neighborhoods" emerged.
  • Real-time Potential: Because the SOM functions as a look-up on a pre-trained grid, it can provide quality estimates instantaneously as network conditions change, allowing for immediate provider intervention.

Conclusion and Future Work

This pilot study demonstrates that Kohonen Maps are a viable tool for bridging the QoS-QoE gap in IPTV services. By simplifying outputs into intuitive clusters, the model provides actionable data for network management. Future work will focus on expanding the sequence database, incorporating a wider variety of scenes, and investigating the integration of additional objective metrics to further boost prediction accuracy, particularly for 4K/UHD resolutions.

Summary updated

Abstract

Abstract This paper deals with an analysis of Kohonen map usage possibility for real-time evaluation of end-user video quality perception. The Quality of Service framework (QoS) describes how the network impairments (network utilization or packet loss) influence the picture quality, but it does not reflect precisely on customer subjective perceived quality of received video stream. There are several objective video assessment metrics based on mathematical models trying to simulate human visual system but each of them has its own evaluation scale. This causes a serious problem for service providers to identify a critical point when intervention into the network behaviour is needed. On the other hand, subjective tests (Quality of Experience concept) are time-consuming and costly and of course, cannot be performed in real-time. Therefore, we proposed a mapping function able to predict subjective end-user quality perception based on the situation in a network, video stream features and results obtained from the objective video assessment method.
BibTeX
@inproceedings{frnda2020qos,
  author = {Jaroslav Frnda and Marek Ďurica and Mihail Savrasovs and Philippe Fournier‐Viger and Jerry Chun‐Wei Lin},
  title = {{QoS to QoE Mapping Function for Iptv Quality Assessement Based on Kohonen Map: A Pilot Study}},
  booktitle = {Transport},
  year = {2020},
  doi = {10.2478/ttj-2020-0014},
}
Figure