Forecasting Lightpath Quality of Transmission and Implementing Uncertainty in the Forecast Models

Somaiieh Yousefi, Hussein Chouman, Petar Djukic, Firouzeh Golaghazadeh, Christine Tremblay, Christian DesrosiersJournal of Lightwave Technology 2023 · DOI

In the world of high-speed optical networks, certainty is expensive. To ensure that data keeps flowing despite fiber aging or equipment degradation, operators bake in massive "system margins"—essentially safety buffers of signal quality that sit idle, wasting potential bandwidth.

A recent paper published in the Journal of Lightwave Technology by Yousefi et al. proposes a way to reclaim that lost capacity. By using deep learning to forecast the Quality of Transmission (QoT) and—more importantly—quantifying exactly how much we should trust those forecasts, the researchers are paving the way for more efficient, "margin-less" optical networks.

TL;DR

Researchers developed a deep learning framework to forecast signal quality in optical fibers while providing reliable "confidence intervals" for those predictions. By using Monte Carlo Dropout and Uncertainty Calibration, they created a model that doesn't just predict the future, but tells operators exactly how much risk they are taking by lowering their safety margins.

What Problem Does This Paper Address?

Optical network operators are currently stuck in a "better safe than sorry" loop. Because they can't perfectly predict how a lightpath’s signal-to-noise ratio (SNR) will fluctuate over months or years, they add a mandatory signal buffer (margin). This margin ensures service isn't interrupted, but it also means a significant portion of the fiber's theoretical bandwidth is never used.

To lower these margins, operators need to forecast future QoT. However, a single "point prediction" (e.g., "The SNR will be 12dB tomorrow") isn't enough. If the model is wrong and the operator has lowered the margin, the connection drops. Operators need probabilistic forecasting: they need to know the range of possible outcomes and the likelihood of the worst-case scenario.

Methods

The authors compared four forecasting approaches:

  1. Naive Model: A baseline that assumes the next value will be the same as the last seen value.
  2. Multilayer Perceptron (MLP): A standard feed-forward neural network.
  3. Long Short-Term Memory (LSTM): A network designed specifically for sequential data.
  4. N-Beats: A sophisticated deep learning architecture based on residual links, originally designed for generic time-series forecasting.

To move from simple predictions to uncertainty estimation, they employed two key techniques:

  • Monte Carlo (MC) Dropout: During the "inference" phase (when the model is making predictions), they randomly "drop" neurons. By running the model 100+ times for the same input, they generate a distribution of answers. The spread of these answers represents the model's uncertainty.
  • Deep Quantile Regression: Used as a baseline, this trains the model to predict specific percentiles (like the 5th and 95th) rather than just the average.
  • Uncertainty Calibration (UCE): Raw uncertainty from a neural network is often "uncalibrated"—the model might be overconfident. The authors used Expected Uncertainty Calibration Error (UCE) to tune the model so that its predicted confidence actually matches its observed error rate.

Datasets

The study used real-world field data:

  • NASP (North American Service Provider) Dataset: This contains Bit Error Rate (BER) measurements from 140 channels, sampled at 15-minute intervals.
  • The final processed dataset included over 5,000 data windows, split into training (70%), validation (15%), and testing (15%) sets.

Key Insights and Ideas

  • The "Naive" Paradox: Interestingly, for stable optical channels, the simple Naive model often outperformed complex deep learning models in terms of raw accuracy (RMSE). This highlights that in stable environments, "more of the same" is a hard baseline to beat.
  • Uncertainty is a Requirement, Not a Feature: The authors argue that for industrial applications, a model's confidence is just as important as its accuracy. An accurate model that can't tell you when it's guessing is useless for risk management.
  • Calibration is the Secret Sauce: The paper shows that raw MC Dropout results are often skewed (asymmetric). By applying UCE calibration and optimizing dropout rates (finding that 0.05% to 0.01% worked best), they transformed a "jittery" prediction into a symmetrical, reliable confidence interval.

Key Results

  • Top Performer: The MLP model was identified as the best deep learning candidate for this specific dataset, achieving a very low Root Mean Square Error (RMSE) of approximately 0.06 dB for a 1-hour forecast.
  • Calibration Success: Before calibration, the uncertainty bounds were often far from the true values. After applying UCE-based optimization, the researchers produced a model where the predicted variance closely correlated with the actual mean squared error.
  • Optimal Architecture: The best results for uncertainty came from an MLP with 128 neurons per layer, using a tiny amount of dropout (0.1%) during training and even less (0.01%) during testing.

Limitations and Future Work

  • Stable Data Bias: The NASP dataset used in this study did not exhibit strong seasonality or trends. The authors note that the Naive model's success might not hold up in more volatile networks where traffic patterns shift dramatically.
  • Computational Cost: Running a model 500 times (variational inferences) to get one prediction is computationally expensive. While they optimized this, real-time deployment on thousands of channels simultaneously would require significant hardware resources.
  • Future Directions: The authors suggest exploring how these uncertainty-aware models can be integrated directly into network control systems to automate the dynamic adjustment of system margins in real-time.

Summary updated

Abstract

The recent popularity of using deep learning models for the forecasting of time series calls for methods to not only predict the target but also measure the uncertainty of the prediction accurately. Working with time series requires reliable and stable forecasters. An essential component of the reliability of machine learning (ML) and deep learning (DL) models is the estimation of the uncertainty. In this work, we address building and characterizing time series forecasters, including N-Beats, Long Short-Term Memory (LSTM) and Multilayer Perceptron (MLP) against the Naive model, and define the confidence margins, and uncertainty for the selected model. All the implementations are conducted in Python programming language. Random sampling is performed to avoid overfitting. Our target field data is North American Service Provider data sets (NASP). Among the implemented models, the MLP model is selected to measure the uncertainty and confidence level, and the Monte Carlo dropout, which approximates Bayesian uncertainty, is applied during inference to render the implementation of uncertainty calculations. Quantile Regression is also implemented on the MLP algorithm as a baseline to predict the confidence intervals and to evaluate our strategy for estimating uncertainty. To establish reliable uncertainty estimation in time series predictions, we performed uncertainty calibration. Motivated by recent developments in Expected Uncertainty Calibration Error (UCE), we modified the uncertainty calculated by the probabilistic Bayesian estimations. Detailed experiments and architectures of the solution are presented.
BibTeX
@article{yousefi2023forecasting,
  author = {Somaiieh Yousefi and Hussein Chouman and Petar Djukic and Firouzeh Golaghazadeh and Christine Tremblay and Christian Desrosiers},
  title = {{Forecasting Lightpath Quality of Transmission and Implementing Uncertainty in the Forecast Models}},
  journal = {Journal of Lightwave Technology},
  year = {2023},
  doi = {10.1109/jlt.2023.3252441},
}
Figure