Scalable Optimization for Large-Scale Infrastructure Networks

52 concepts54 papers4 proposals

Modern society relies on massive, interconnected networks—from electrical grids and water distribution systems to metropolitan camera feeds and cellular infrastructure. Managing these systems requires solving complex resource allocation, scheduling, and state estimation problems in real time. However, as highlighted by Yih-Fang Huang et al. in [Introduction to the Issue on Signal Processing in Smart Electric Power Grid], traditional optimization solvers scale poorly, with computational requirements increasing quadratically or exponentially relative to the number of nodes.

Why does this happen? Infrastructure networks are governed by physical laws (such as non-linear power-flow or hydraulic equations) and strict operational constraints. When transitioning from small benchmark systems to real-world networks containing thousands of nodes, the combination of non-convexity and high dimensionality creates severe performance bottlenecks, preventing real-time responsiveness (James D. McCalley et al. [Auto-steered Information-Decision Processes for Electric System Asset Management]; Ye Shi et al. [Mixed integer nonlinear programming for Joint Coordination of Plug-in Electrical Vehicles Charging and Smart Grid Operations]).

To overcome this "curse of dimensionality," recent research focuses on four core strategies: linearization, decomposition, spectral initialization, and learning-based surrogates.

Linearization and Convex Relaxations

One of the most direct ways to bypass the computational burden of non-convex optimization is to reshape the mathematical landscape entirely. In power systems, for example, the Alternating Current Optimal Power Flow (AC-OPF) problem is notoriously difficult because voltage and phase relationships are inherently non-linear.

To tackle this, Swaroop S. Guggilam et al. [Scalable Optimization Methods for Distribution Networks With High PV Integration] demonstrate that linearizing complex power-flow equations can transform a non-convex problem into a fast, convex alternative. By flattening the optimization landscape, solvers can handle thousands of nodes in seconds while maintaining high accuracy. Similarly, Pengchao Tian et al. [Two-Stage Optimization for Efficient V2G Coordination in Distribution Power System] combine a "Difference of Convex" relaxation with a trust-region search. This approach achieves near-optimal results for vehicle-to-grid coordination over 98% faster than standard industry solvers, illustrating how relaxation techniques make large-scale deployment computationally viable.

Decomposition and Hierarchical Control

When a network is too massive to solve centrally, the logical alternative is to break it apart. Decomposition techniques distribute the computational load across space or hierarchy, solving localized sub-problems that coordinate via a global master problem.

For instance, V. Causevic [Information-constrained Optimal Control of Distributed Systems with Power Constraints] shows that by reformulating global control problems into a covariance selection framework and applying dual decomposition, a complex global network can be decoupled into independent, manageable sub-problems.

In a similar vein, Derek Chang et al. [Stochastic Resource Allocation for Electricity Distribution Network Resilience] utilize Benders decomposition paired with a greedy heuristic to navigate disaster recovery modeling. By isolating master strategic decisions (like asset placement) from sub-problem operational schedules (like post-storm repairs), they achieve rapid convergence on large-scale infrastructure. Hierarchical structures also allow multi-scale coordination, as seen in the work of Fuxing Zhang et al. [Optimizing Hierarchical Power Distribution of Multiple Local Energy Network Systems in Grid-Connected Mode], which coordinates local, neighborhood, and wide-area energy networks to maintain real-time responsiveness without overloading a single central solver.

[Global Master Problem]
       /       \
      v         v
[Sub-Problem 1] [Sub-Problem 2]  --> Solved in parallel

Spectral Initialization and Global Certificates

Even if a non-convex problem cannot be easily relaxed or decomposed, we can drastically speed up convergence by starting the solver in the right neighborhood. Standard iterative solvers often get trapped in "spurious" or fake local minima if initialized poorly, wasting valuable compute time.

Iven Guzel [Power System State Estimation by Phase Synchronization and Eigenvectors] addresses this by reformulating non-convex state estimation as a phase synchronization problem. This mathematical shift provides a high-quality spectral initialization—a mathematically grounded "best guess"—alongside a global optimality certificate. By ensuring the solver starts close to the true global optimum, the computational overhead of navigating complex non-convex landscapes is minimized, making real-time monitoring feasible for networks with tens of thousands of nodes.

Learning-Based Surrogates and Hybrid Heuristics

Recently, researchers have turned to machine learning and hybrid heuristics to bypass traditional mathematical optimization bottlenecks entirely. Instead of repeatedly executing expensive physics-based iterations, these methods learn the underlying patterns of the network to predict solutions instantly.

Fuat Can Beylunioglu et al. [Robust Training for AC-OPF (Student Abstract)] propose a robust training loop for neural networks managing power flows. By using an iterative feedback loop to identify and generate targeted data for regions where the model struggles, they bypass traditional solver overhead while maintaining high accuracy. Furthermore, when dealing with massive synthetic networks up to 30,000 buses, the ARPA-E Grid Optimization Competition challenges have demonstrated that the most effective solutions combine advanced optimization theory with high-performance computing and hybrid heuristics to manage extreme complexity (Stephen T. Elbert et al. [ARPA-E Grid Optimization (GO) Competition Challenge 2]).

Key Foundational Work

Impact of interference on multi-hop wireless network performance (2003) — contributed 1 concept elements to this direction.
Auto-steered Information-Decision Processes for Electric System Asset Management (2006) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_large_scale_network_optimization_20ee0a39,
  title = {large-scale network optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/large-scale-network-optimization/rdr_20ee0a39a7ec4407924393d1589fdaea},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_large_scale_network_optimization_20ee0a39_v1,
  title = {large-scale network optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/large-scale-network-optimization/rdr_20ee0a39a7ec4407924393d1589fdaea/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake