As a critical technology in network virtualization, virtual network embedding (VNE) focuses on how to allocate physical resources to virtual network requests efficiently. Because the VNE problem is NP-hard, most of the existing approaches are heuristic-based algorithms that tend to converge to a local optimal solution and have a low performance. In this paper, we propose an algorithm that combines the basic Monte Carlo tree search (MCTS) method with node importance to apply domain-specific knowledge. For a virtual network request, we first model the embedding process as a finite Markov decision process (MDP), where each virtual node is embedded in one state in the order of node importance that we design. The shortest-path algorithm is then applied to embed links in the terminal state and return the cost as a part of the reward. Due to the reward delay mechanism of the MDP, the result of link mapping can affect the action selected in the previous node mapping stage, coordinating the two embedding stages. With node importance, domain-specific knowledge can be used in the Expansion and Simulation stages of MCTS to speed up the search and estimate the simulation value more accurately. The experimental results show that, compared with the existing classic algorithms, our proposed algorithm can improve the performance of VNE in terms of the average physical node utilization ratio, acceptance ratio, and long-term revenue to cost ratio.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Bashir AK, Arul R, Basheer S, Raja G, Jayaram R, Qureshi NMF (2019) An optimal multitier resource allocation of cloud RAN in 5G using machine learning. Trans Emerg Telecommun Tech 30(8):e3627
Bashir AK, Ohsita Y, Murata M (2015) Abstraction layer based distributed architecture for virtualized data centers
Chowdhury NMMK, Boutaba R (2009) Network virtualization: state of the art and research challenges. IEEE Commun Mag 47(7):20–26
Mosharaf Kabir Chowdhury NM, Rahman MR, Boutaba R (2009) Virtual network embedding with coordinated node and link mapping. In: IEEE INFOCOM 2009. IEEE, pp 783–791
Wang C, Shanbhag S, Tilman W (2012) Virtual network mapping with traffic matrices. In: IEEE international conference on communications (ICC). IEEE, pp 2717–2722
Cao H, Yang L, Liu Z, Wu M (2016) Exact solutions of vne: a survey. China Commun 13(6):48–62
Cao H, Hu H, Qu Z, Yang L (2018) Heuristic solutions of virtual network embedding: a survey. China Commun 15(3):186–219
Zhang P, Yao H, Liu Y (2016) Virtual network embedding based on the degree and clustering coefficient information. IEEE Access 4:8572–8580
Cheng X, Su S, Zhang Z, Wang H, Yang F, Luo Y, Wang J (2011) Virtual network embedding through topology-aware node ranking. Comput. Commun. Rev 41(2):39–47
Cao H, Zhu Y, Yang L, Zheng G (2017) A efficient mapping algorithm with novel node-ranking approach for embedding virtual networks. IEEE Access 5(1):22054–22066
Zhang P, Yao H, Liu Y (2018) Virtual network embedding based on computing, network, and storage resource constraints. IEEE Internet Things J. 5(5):3298–3304
Zhang P, Yao H, Qiu C, Liu Y (2018) Virtual network embedding using node multiple metrics based on simplified ELECTRE method. IEEE Access 6:37314–37327
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction MIT press
Bashir AK, Ohsita Y, Murata M (2016) Abstraction layer based virtual data center architecture for network function chaining
Korf RE (1985) Depth-first iterative-deepening, vol 27
Swiechowski M, Mandziuk J (2014) Self-adaptation of playing strategies in general game playing, vol 6
Haeri S, Trajković L (2018) Virtual network embedding via Monte Carlo tree search. IEEE Trans Cybern. 48(2):510–521
Soualah O, Fajjari I, Aitsaadi N, Mellouk A (2015) A batch approach for a survivable virtual network embedding based on monte-carlo tree search. In: IFIP/IEEE international symposium on integrated network management (IM). IEEE, pp 36– 43
Yao H, Chen X, Li M, Zhang P, Wang L (2018) A novel reinforcement learning algorithm for virtual network embedding. Neurocomputing 284:1–9
Schadd M.P.D., Winands M.H.M., van den Herik H.J., Chaslot G.M.J.B., Uiterwijk J.W.H.M. (2008) Single-Player Monte-Carlo Tree Search. In: van den Herik H.J., Xu X., Ma Z., Winands M.H.M. (eds), vol 5131. Computers and Games. CG 2008. Lecture Notes in Computer Science Springer, Berlin, Heidelberg, pp 1–12
Fischer A, Botero JF, Beck MT, De Meer H, Hesselbach Xavier (2013) Virtual network embedding: a survey. IEEE Commun Surveys Tuts 15(4):1888–1906
Kennedy J (2010). In: Sammut C., Webb G. I. (eds) Particle Swarm Optimization. Springer US, Boston, MA, pp 760–766. isbn:978-0-387-30164-8
Dorigo M, Birattari M (2010) Ant colony optimization. Springer, Berlin
Goldberg DE (2006) Genetic algorithms. Pearson Education India, Bengaluru
Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220 (4598):671–680
Wang C, Yian S, Zhou L, Peng S, Yuan Y, Huang H (2016) A virtual network embedding algorithm based on hybrid particle swarm optimization. In: Int conf smart comput commun, pp 568–576
Mijumbi R, Gorricho J-L, Serrat J, Claeys M, De Turck F, Latré S (2014) Design and evaluation of learning algorithms for dynamic resource management in virtual networks. In: 2014 IEEE netw oper manag symp, pp 1–9
Yao H, Zhang B, Zhang P, Wu S, Jiang C, Guo S (2018) RDAM: a reinforcement learning based dynamic attribute matrix representation for virtual network embedding. IEEE trans Emerg Top comput
Yu M, Yi Y, Rexford J, Chiang M (2008) Rethinking virtual network embedding: substrate support for path splitting and migration. ACM SIGCOMM Comput Commun Rev 38(2):17–29
Kocsis L, Szepesvári C (2006) Bandit based Monte-Carlo planning
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, et al. (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484
Floyd RW (1962) Algorithm 97: shortest path. Commun. ACM 5(6):345
Zhang Z, Cheng X, Su S, Wang Y, Shuang K, Luo Y (2013) A unified enhanced particle swarm optimization-based virtual network embedding algorithm. Int J Commun Syst 26(8):1054–1073
Calheiros RN, Ranjan R, Beloglazov A, De Rose CAF, Buyya R (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw Pract Exp 41(1):23–50
This work was supported by the National Natural Science Foundation of China under Grant Nos. 61702089, the basic scientific research operating fund of central universities under Grant No. N182304021, and the scientific research plan for institutions of higher learning of Hebei province under Grant No. ZD2019306.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Guangcong Zheng and Cong Wang contributed equally to this work and should be considered co-first authors.
About this article
Cite this article
Zheng, G., Wang, C., Shao, W. et al. A single-player Monte Carlo tree search method combined with node importance for virtual network embedding. Ann. Telecommun. (2020). https://doi.org/10.1007/s12243-020-00772-5
- Network virtualization
- Virtual network embedding
- Reinforcement learning
- Markov decision process
- Monte Carlo tree search
- Node ranking