$β$-Divergence-Based Latent Factorization of Tensors model for QoS prediction

Zemiao Peng, Hao Wu

TL;DR

This paper introduces $\beta$-NLFT, a tensor factorization model that improves Quality-of-Service (QoS) prediction by replacing standard Euclidean distance with the more flexible $\beta$-divergence objective function. To make the model practical, the authors integrate Particle Swarm Optimization (PSO) to automatically tune hyperparameters, resulting in significantly higher accuracy compared to current state-of-the-art methods.

What Problem Does This Paper Address?

In the world of Web services, choosing the right provider often depends on non-functional "Quality-of-Service" (QoS) metrics like response time and throughput. However, collecting this data is expensive and time-consuming, leading to datasets that are High-Dimensional and Incomplete (HDI).

Furthermore, QoS isn't static; it fluctuates over time based on network conditions and server load. While Non-negative Latent Factorization of Tensors (NLFT) is great at capturing these temporal patterns, most existing models rely on Euclidean distance to measure error. The authors argue that Euclidean distance is just a special case ($\beta=2$) of the broader $\beta$-divergence family. By restricting themselves to $\beta=2$, existing models miss out on the potential accuracy gains offered by other $\beta$ values that might better fit the underlying distribution of QoS data.

Methods

The authors propose the $\beta$-NLFT model, which stands on three main pillars:

  1. Tensor Factorization with Biases: The model represents dynamic QoS data as a 3D tensor (User $\times$ Service $\times$ Time). It decomposes this tensor into latent factor matrices while incorporating linear bias vectors to account for individual user and service characteristics.
  2. $\beta$-Divergence Objective: Instead of minimizing the squared difference (Euclidean), the model minimizes the $\beta$-divergence between observed and predicted values. This allows the model to adapt to different data characteristics by varying the $\beta$ parameter (e.g., $\beta=1$ corresponds to generalized Kullback-Leibler divergence).
  3. Self-Adaptive Hyperparameters via PSO: Tuning parameters like $\beta$ and regularization coefficients ($\lambda$) manually is a "thorny issue." The authors embed a Particle Swarm Optimization (PSO) algorithm to automatically search for the optimal hyperparameters during the learning process, improving the model's usability and performance.

The model is optimized using a Single Latent Factor-dependent, Non-negative and Multiplicative Update on the Tensor (SLF-NMUT) algorithm, ensuring that the resulting factors remain non-negative (as QoS values are always positive).

Datasets

The researchers evaluated their model using two real-world dynamic QoS datasets collected by WSMonitor:

  • D1 (Response-Time): Measures how long a service takes to respond (0–20s).
  • D2 (Throughput): Measures data transfer rates (0–1,000kbps).

Both datasets involve 142 users, 4,532 services, and 64 time slots, resulting in over 30 million potential data points, though only a fraction are observed.

Key Insights and Ideas

  • Generalization is Key: By treating the objective function as a variable ($\beta$) rather than a constant (Euclidean), the model can "shape" its error tolerance to match the specific noise and distribution of the dataset.
  • The "Aha" of Automation: Hyperparameter tuning is often the "dark art" of machine learning. Using PSO to treat hyperparameters as a moving swarm that converges on an optimal configuration removes the need for exhaustive (and slow) grid searches.
  • Temporal Awareness: By using tensors instead of matrices, the model inherently preserves the "time" dimension, which is crucial for predicting QoS in volatile network environments.

Key Results

The $\beta$-NLFT model was compared against several state-of-the-art models (HDOP, TCA, and BNLFT). The results showed clear dominance:

  • On Dataset D2 (Throughput): $\beta$-NLFT achieved an RMSE of 18.474, which is 40.26% lower than the HDOP model and 1.73% lower than the previous best-in-class BNLFT model.
  • Mean Absolute Error (MAE): On the same dataset, the MAE was 3.125, outperforming the closest competitor by roughly 5%.
  • Consistency: The model showed similar performance gains across both response-time and throughput datasets, proving its versatility.

Limitations and Future Work

While the results are promising, the authors identify a few areas for growth:

  • Evolutionary Algorithms: They suggest that even more advanced evolutionary algorithms could further refine the hyperparameter self-adaptation process.
  • Objective Function Expansion: There is interest in extending the objective function even further to incorporate other types of divergences or constraints to boost performance in even noisier environments.
  • Scalability: While not explicitly detailed as a failure, the computational overhead of running a PSO "swarm" alongside tensor factorization is a factor that might need optimization for even larger-scale industrial applications.

Summary updated

Abstract

A nonnegative latent factorization of tensors (NLFT) model can well model the temporal pattern hidden in nonnegative quality-of-service (QoS) data for predicting the unobserved ones with high accuracy. However, existing NLFT models' objective function is based on Euclidean distance, which is only a special case of $β$-divergence. Hence, can we build a generalized NLFT model via adopting $β$-divergence to achieve prediction accuracy gain? To tackle this issue, this paper proposes a $β$-divergence-based NLFT model ($β$-NLFT). Its ideas are two-fold 1) building a learning objective with $β$-divergence to achieve higher prediction accuracy, and 2) implementing self-adaptation of hyper-parameters to improve practicability. Empirical studies on two dynamic QoS datasets demonstrate that compared with state-of-the-art models, the proposed $β$-NLFT model achieves the higher prediction accuracy for unobserved QoS data.
BibTeX
@misc{peng2022divergence,
  author = {Zemiao Peng and Hao Wu},
  title = {{$β$-Divergence-Based Latent Factorization of Tensors model for QoS prediction}},
  year = {2022},
  eprint = {2208.06778},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2208.06778},
}
Figure