Abstract
Virtual Network Embedding (VNE) is a fundamental challenge in network virtualization, requiring the efficient mapping of virtual network requests (VNRs) onto shared physical infrastructure. Existing reinforcement learning (RL) solutions often suffer from rigid, unidirectional node mapping sequences and poor generalization across varying network scales. We present FlagVNE, a novel framework that addresses these limitations through a bidirectional action-based Markov Decision Process (MDP) and a hierarchical policy architecture. By jointly selecting virtual and physical nodes, FlagVNE expands the search space for optimal embeddings. A hierarchical policy architecture manages the resulting action space, while a curriculum-based meta-learning approach ensures robust performance across different network sizes. Experimental evaluations on standard topologies demonstrate that FlagVNE achieves significant improvements in request acceptance rates and long-term revenue compared to state-of-the-art baselines.
Introduction
In the era of 5G and cloud computing, Network Virtualization (NV) enables multiple isolated virtual networks to coexist on a common physical substrate. The core efficiency of this paradigm hinges on the Virtual Network Embedding (VNE) problem: mapping virtual nodes and links to physical counterparts while satisfying resource constraints such as CPU, storage, and bandwidth.
VNE is an NP-hard combinatorial problem. While traditional heuristics and recent RL-based methods have been proposed, they typically follow a "unidirectional" mapping order—fixing the sequence of virtual nodes before placement. This rigidity often leads to sub-optimal solutions. Furthermore, models trained on specific network sizes frequently fail to generalize to larger or more complex requests.
The FlagVNE Framework
FlagVNE introduces three primary innovations to overcome the limitations of existing VNE solvers.
Bidirectional Action-based MDP
Unlike unidirectional approaches, FlagVNE employs a bidirectional action design. At each step, the agent simultaneously selects a virtual node and a physical node. This approach allows the agent to dynamically prioritize the most critical virtual nodes based on the current state of the physical network, significantly expanding the search space and improving the potential for finding optimal mappings.
Hierarchical Policy Architecture
To handle the increased complexity of joint node selection, FlagVNE utilizes a hierarchical decoder. A high-level ordering policy identifies the next virtual node, while a low-level placement policy determines the physical host. This architecture reduces the action space complexity from multiplicative to additive, ensuring computational efficiency.
Meta-RL with Curriculum Scheduling
To achieve broad generalization, FlagVNE incorporates Model-Agnostic Meta-Learning (MAML). This allows the model to learn a "meta-policy" that can be rapidly adapted to size-specific sub-policies. To stabilize training, a curriculum scheduling strategy is introduced: the agent is first trained on smaller, simpler VNRs and gradually progresses to larger tasks as its confidence (measured by policy entropy) increases.
Experimental Results
The framework was evaluated on the GEANT and WX100 physical network benchmarks using 1,000 Poisson-arriving VNRs. FlagVNE demonstrated superior performance across all key metrics:
- Acceptance Rate: Surpassed the best RL baseline (A3C-GCN) by 10.4% on the GEANT topology.
- Revenue: Improved Long-term Average Revenue (LAR) by up to 12.8% on the WX100 topology.
- Resource Efficiency: Achieved a Revenue-to-Cost (R2C) ratio gain of up to 45.1% over traditional heuristics.
Conclusion
FlagVNE represents a significant step forward in automated network management. By combining bidirectional decision-making with meta-learning and curriculum strategies, it provides a flexible and highly generalizable solution for the VNE problem. Future work will focus on integrating link routing directly into the RL process and enhancing scalability for ultra-large-scale data center networks.
Summary updated