Experiments with Neural Networks for Real Time Implementation of Control

Peter K. Campbell, Michael Dale, Herman L. Ferrá, Adam KowalczykNeurIPS 1995

TL;DR: Researchers at Telstra Research Laboratories have developed a two-stage neural network architecture to handle real-time bandwidth allocation in telecommunications networks. By training "mask perceptrons" to mimic slow but optimal linear programming solvers, they achieved up to 99% of optimal throughput with execution times fast enough for millisecond-level network restoration.

The Challenge: Real-Time Bandwidth Allocation

In modern fiber-optic networks—specifically Synchronous Digital Hierarchy (SDH)—bandwidth must be dynamically allocated to different paths to maximize throughput. When a link fails or traffic spikes, the network needs to reconfigure itself almost instantly, ideally in under 60 milliseconds.

The "gold standard" for finding the best configuration is Linear Programming (LP). However, LP is computationally expensive; for large networks, it can take anywhere from seconds to hours to find a solution. While heuristic algorithms are faster, they often produce sub-optimal results. The authors sought a "middle way": a system that offers the near-optimal performance of LP with the lightning-fast execution required for real-time control.

Proposed Architecture: A Two-Stage Controller

To bridge the gap between speed and optimality, the authors proposed a two-stage controller architecture:

  1. Stage 1 (Approximation): A feedforward neural network called a mask perceptron takes the current network state (link capacities and traffic demand) and produces an approximate allocation.
  2. Stage 2 (Refinement): Because the first stage might produce "illegal" solutions (e.g., non-integers or exceeding physical link limits), a second stage cleans up the results. The authors tested two versions of this refinement stage:
  3. Neural-Heuristic: A rule-based algorithm that adjusts capacities to satisfy constraints.
  4. Neural-Recurrent: A second mask perceptron running in a recurrent loop, iteratively nudging the solution toward a valid state.
Network topology and the corresponding mask perceptron architecture.

The Mask Perceptron: Unlike standard deep learning models, the mask perceptron uses binary threshold units and "AND" gates. This is a critical design choice because it requires no multiplications—only comparisons and additions—making it extremely efficient for implementation on hardware like Digital Signal Processors (DSPs).

Training and Datasets

The researchers used a "teacher-student" training approach on a simulated inter-city SDH network. The dataset consisted of 4,000 total instances (2,000 for training, 2,000 for testing).

The network inputs included 13 link capacities and 42 traffic demand values, while the outputs were 126 integer values representing path capacity adjustments. The "ground truth" labels for training were generated using a classical LP optimiser acting as the "teacher."

Key Insights

  • Learning the Optimizer: Instead of trying to solve the non-linear optimization problem from scratch, the neural network is taught to replicate the behavior of a slow, optimal algorithm. This allows the system to "distill" complex logic into a fast execution format.
  • Hardware-Friendly AI: The choice of the mask perceptron architecture highlights a focus on deployment. By avoiding multiplications and non-linear transformations, the model is tailor-made for the low-latency constraints of telecommunications hardware.
  • The "Amplification" Trick: In the recurrent stage, the authors found that small errors were often lost due to rounding. They introduced an "error amplification" factor to boost these signals, allowing the network to "see" and correct minor constraint violations.

Results and Performance

The neural network approaches demonstrated a massive speed advantage, generating solutions in roughly 45 to 56 milliseconds. In contrast, the traditional LP solver took between 2 and 60 seconds.

Throughput performance comparison across different solution types.

Beyond speed, the models maintained high accuracy and robustness:

  • Accuracy: The Feedforward/Heuristic combination achieved 99.08% of the optimal LP throughput on the training set and 98.90% on the test set. The Recurrent version achieved roughly 95%.
  • Robustness: The neural-heuristic approach was particularly effective at finding high-quality solutions for heavily degraded networks (where links were partially or completely cut), outperforming standalone heuristic methods like the Gopal algorithm.

Limitations and Future Work

While promising, the recurrent neural network failed to find a valid (violation-free) solution in about 20% of the test cases. The authors note that while the recurrent approach is more "generic" and less hand-tuned than the heuristic, it requires further refinement to be fully reliable.

Additionally, these experiments were conducted on a specific inter-city network topology. Future work will investigate how this architecture scales to more complex and larger SDH architectures, and whether further training of the recurrent stage can close the performance gap with the heuristic-based refinement.

Summary updated

BibTeX
@inproceedings{campbell1995experiments,
  author = {Peter K. Campbell and Michael Dale and Herman L. Ferrá and Adam Kowalczyk},
  title = {{Experiments with Neural Networks for Real Time Implementation of Control}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1995},
}
Figure