## Abstract

Network softwarization has recently been enabled via the software-defined networking (SDN) paradigm, which separates the data plane from control plane allowing for a flexible and centralized control of networks. This separation facilitates implementation of machine learning techniques for network management and optimization. In this work, a machine learning-based multipath routing (MLMR) framework is proposed for software-defined networks with quality-of-service (QoS) constraints and flow rules space constraints. The QoS-aware multipath routing problem in SDN is modeled as multicommodity network flow problem with side constraints, that is known to be NP-hard. The proposed framework utilizes network status estimates, and their corresponding routing configurations available at the network central controller to learn a mapping function between them. Once the mapping function is learned, it is applied on live-inputs of network status and routing requests to predict a multipath routing solutions in real-time. Performance evaluations of the MLMR framework on real traces of network traffic verify its accuracy and resilience to noise in training data. Furthermore, the MLMR framework demonstrates more than 98.99% improvement in computational efficiency.

This is a preview of subscription content, access via your institution.

## References

- 1.
Cisco, V.: Cisco Visual Networking Index: Forecast and Trends, 2017–2022, vol. 1. White Paper, (2018)

- 2.
Nunes, B.A.A., Mendonca, M., Nguyen, X.-N., Obraczka, K., Turletti, T.: A survey of software-defined networking: past, present, and future of programmable networks. IEEE Commun. Surv. Tutor.

**16**(3), 1617–1634 (2014) - 3.
Xie, J., Yu, F.R., Huang, T., Xie, R., Liu, J., Wang, C., Liu, Y.: A survey of machine learning techniques applied to software defined networking (sdn): research issues and challenges. IEEE Commun. Surv. Tutor.

**21**(1), 393–430 (2019). First quarter - 4.
Latah, M., Toker, L.: Artificial intelligence enabled software defined networking: a comprehensive overview. IET Netw. 11 (2018)

- 5.
Yanjun, L., Xiaobo, L., Osamu, Y.: Traffic engineering framework with machine learning based meta-layer in software-defined networks. In Proceedings of the Network Infrastructure and Digital Content (IC-NIDC), 2014 4th IEEE International Conference on. IEEE, pp. 121–125 (2014)

- 6.
Azzouni, A., Boutaba, R., Pujolle, G.: Neuroute: Predictive dynamic routing for software-defined networks.

*arXiv preprint*. arXiv:1709.06002 (2017) - 7.
Francois, F., Gelenbe, E.: Optimizing secure sdn-enabled inter-data centre overlay networks through cognitive routing. In Proceedings of the 2016 IEEE 24th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 283–288 (2016)

- 8.
Lin, S., Akyildiz, I. F., Wang, P., Luo, M.: Qos-aware adaptive routing in multi-layer hierarchical software defined networks: a reinforcement learning approach. In Proceedings of the 2016 IEEE International Conference on Services Computing (SCC), pp. 25–33 (2016)

- 9.
Stampa, G., Arias, M., Sánchez-Charles, D., Muntés-Mulero, V., Cabellos, A.: A deep-reinforcement learning approach for software-defined networking routing optimization.

*arXiv preprint*. arXiv:1709.07080 (2017) - 10.
Mendiola, A., Astorga, J., Jacob, E., Higuero, M.: A survey on the contributions of software-defined networking to traffic engineering. IEEE Commun. Surv. Tutor.

**19**(2), 918–953 (2017) - 11.
Wang, Y.-C., Lin, Y.-D., Chang, G.-Y.: Sdn-based dynamic multipath forwarding for inter–data center networking. Int. J. Commun. Syst. 32(1), e3843 (2019). https://onlinelibrary.wiley.com/doi/abs/10.1002/dac.3843

- 12.
He, J., Rexford, J.: Toward internet-wide multipath routing. IEEE Netw.

**22**(2), 16–21 (2008) - 13.
Doshi, M., Kamdar, A.: Multi-constraint qos disjoint multipath routing in sdn. In Moscow Workshop on Electronic and Networking Technologies (MWENT), pp. 1–5 (2018)

- 14.
Zhang, P., Gang, Y., Huang, X., Zeng, S., Xie, K.: Bandwidth allocation with utility maximization in the hybrid segment routing network. IEEE Access

**7**, 85 253–85 261 (2019) - 15.
Gao, C., Wang, H., Zhai, L., Yi, S., Yao, X.: Optimizing routing rules space through traffic engineering based on ant colony algorithm in software defined network. In Proceedings of the 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI), pp. 106–112 (2016)

- 16.
Holmberg, K., Yuan, D.: A multicommodity network-flow problem with side constraints on paths solved by column generation. INFORMS J. Comput. 15(1), pp. 42–57 (2003). https://pubsonline.informs.org/doi/abs/10.1287/ijoc.15.1.42.15151

- 17.
Misra, S., Xue, G., Yang, D.: Polynomial time approximations for multi-path routing with bandwidth and delay constraints. In

*Proceedings of the IEEE INFOCOM 2009*, pp. 558–566 (2009) - 18.
Rauch, H.E., Winarske, T.: Neural networks for routing communication traffic. IEEE Control Syst. Mag.

**8**(2), 26–31 (1988) - 19.
Ouyang, Y.C., Bhatti, A.A.: Neural network-based routing in an intelligent computer communication network. In Proceedings of the System Theory, Twenty-Second Southeastern Symposium on. IEEE, pp. 444–448 (1990)

- 20.
Ali, M.K.M., Kamoun, F.: Neural networks for shortest path computation and routing in computer networks. IEEE Trans. Neural Netw.

**4**(6), 941–954 (1993) - 21.
Park, D.-C., Choi, S.-E.: A neural network based multi-destination routing algorithm for communication network. In 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No. 98CH36227), vol. 2. IEEE, pp. 1673–1678 (1998)

- 22.
Xia, Y., Wang, J.: A discrete-time recurrent neural network for shortest-path routing. IEEE Trans. Autom. Control

**45**(11), 2129–2134 (2000) - 23.
Kato, N., Fadlullah, Z.M., Mao, B., Tang, F., Akashi, O., Inoue, T., Mizutani, K.: The deep learning vision for heterogeneous network traffic control: proposal, challenges, and future perspective. IEEE Wirel. Commun.

**24**(3), 146–153 (2017) - 24.
Wang, M., Cui, Y., Wang, X., Xiao, S., Jiang, J.: Machine learning for networking: workflow, advances and opportunities. IEEE Netw.

**32**(2), 92–99 (2018) - 25.
Fadlullah, Z., Tang, F., Mao, B., Kato, N., Akashi, O., Inoue, T., Mizutani, K.: State-of-the-art deep learning: evolving machine intelligence toward tomorrow’s intelligent network traffic control systems. IEEE Commun. Surv. Tutor. (2017)

- 26.
Boutaba, R., Salahuddin, M.A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., Caicedo, O.M.: A comprehensive survey on machine learning for networking: evolution, applications and research opportunities. J. Internet Serv. Appl.

**9**(1), 16 (2018). https://doi.org/10.1186/s13174-018-0087-2 - 27.
Usama, M., Qadir, J., Raza, A., Arif, H., Yau, K.-L.A., Elkhatib, Y., Hussain, A., Al-Fuqaha, A.: Unsupervised machine learning for networking: techniques, applications and research challenges. IEEE Access

**7**, 65 579–65 615 (2019) - 28.
Choudhury, G., Lynch, D., Thakur, G., Tse, S.: Two use cases of machine learning for sdn-enabled ip/optical networks: traffic matrix prediction and optical path performance prediction [invited]. IEEE/OSA J. Opt. Commun. Netw.

**10**(10), D52–D62 (2018) - 29.
Aibin, M.: Traffic prediction based on machine learning for elastic optical networks. Opt. Switch. Netw. 30, 33–39 (2018). http://www.sciencedirect.com/science/article/pii/S157342771730190X

- 30.
Alvizu, R., Troia, S., Maier, G., Pattavina, A.: Matheuristic with machine-learning-based prediction for software-defined mobile metro-core networks. IEEE/OSA J. Opt. Commun. Netw.

**9**(9), D19–D30 (2017) - 31.
Mao, B., Fadlullah, Z.M., Tang, F., Kato, N., Akashi, O., Inoue, T., Mizutani, K.: Routing or computing? the paradigm shift towards intelligent computer network packet transmission based on deep learning. IEEE Trans. Comput. (2017)

- 32.
Wang, Y., Martonosi, M., Peh, L.-S.: Supervised learning in sensor networks: New approaches with routing, reliability optimizations. In Proceedings of the 3rd Annual IEEE Communications Society on Sensor and Ad Hoc Communications and Networks, vol. 1. IEEE, pp. 256–265 (2006)

- 33.
Martín, I., Hernández, J. A., Troia, S., Musumeci, F., Maier, G., de Dios, O. G.: Is machine learning suitable for solving rwa problems in optical networks? In: Proceedings of the 2018 European Conference on Optical Communication (ECOC). IEEE, pp. 1–3 (2018)

- 34.
Tang, F., Mao, B., Fadlullah, Z.M., Kato, N., Akashi, O., Inoue, T., Mizutani, K.: On removing routing protocol from future wireless networks: a real-time deep learning approach for intelligent traffic control. IEEE Wirel. Commun.

**25**(1), 154–160 (2018) - 35.
Yu, C., Lan, J., Guo, Z., Hu, Y.: Drom: optimizing the routing in software-defined networks with deep reinforcement learning. IEEE Access

**6**, 64 533–64 539 (2018) - 36.
Fang, C., Cheng, C., Tang, Z., Li, C.: Research on routing algorithm based on reinforcement learning in SDN. J. Phys. Conf. Ser.

*1284*, 012053 (2019). https://doi.org/10.1088%2F1742-6596%2F1284%2F1%2F012053 - 37.
Rischke, J., Sossalla, P., Salah, H., Fitzek, F.H.P., Reisslein, M.: Qr-sdn: towards reinforcement learning states, actions, and rewards for direct flow routing in software-defined networks. IEEE Access

**8**, 174 773–174 791 (2020) - 38.
Göransson, P., Black, C., Culver, T.: Chapter 5—the openflow specification. Software Defined Networks (Second Edition) In P. Göransson, C. Black, and T. Culver (eds.) Morgan Kaufmann, Boston, pp. 89 – 136. (2017). http://www.sciencedirect.com/science/article/pii/B9780128045558000053

- 39.
Liu, C., Malboubi, A., Chuah, C.: Openmeasure: Adaptive flow measurement inference with online learning in sdn. In: Proceedings of the 2016 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 47–52 (2016)

- 40.
Suh, J., Kwon, T. T., Dixon, C., Felter, W., Carter, J.: Opensample: a low-latency, sampling-based measurement platform for commodity sdn. In: Proceedings of the 2014 IEEE 34th International Conference on Distributed Computing Systems. IEEE, pp. 228–237 (2014)

- 41.
OpenDylight: Model-Driven Service Adaptation Layer (MD-SAL) (2018). https://docs.opendaylight.org/projects/mdsal

- 42.
Yang, J., Huang, X., Jiang, S.: An accurate approach for traffic matrix estimation in large-scale backbone networks. In: Proceedings of the 2016 15th International Symposium on Parallel and Distributed Computing (ISPDC), pp. 425–431 (2016)

- 43.
Queiroz, W., Capretz, M. A., Dantas, M.: An approach for sdn traffic monitoring based on big data techniques. J. Netw. Comput. Appl. 131, 28–39 (2019). http://www.sciencedirect.com/science/article/pii/S1084804519300244

- 44.
Lakhina, A., Papagiannaki, K., Crovella, M., Diot, C., Kolaczyk, E. D., Taft, N.: Structural analysis of network traffic flows. In: Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems, ser. SIGMETRICS ’04/Performance ’04. Association for Computing Machinery, New York, pp. 61–72 (2004). https://doi.org/10.1145/1005686.1005697

- 45.
Stoev, S., Taqqu, M. S., Park, C., Michailidis, G., Marron, J.: Lass: a tool for the local analysis of self-similarity. Comput. Stat. Data Anal., 50(9), 2447–2471 (2006), statistical signal extraction and filtering. http://www.sciencedirect.com/science/article/pii/S0167947304004062

- 46.
Kirichenko, L., Radivilova, T., Deineko, Z.: Comparative analysis for estimating of the hurst exponent for stationary and nonstationary time series. Inf. Technol. Knowl.

**5**(1), 371–388 (2011) - 47.
Chen, C.: hurst parameter estimate (2020) https://www.mathworks.com/matlabcentral/fileexchange/19148-hurst-parameter-estimate

- 48.
Martín, I., Troia, S., Hernández, J.A., Rodríguez, A., Musumeci, F., Maier, G., Alvizu, R., González de Dios, Ó.: Machine learning-based routing and wavelength assignment in software-defined optical networks. IEEE Trans. Netw. Serv. Manage.

**16**(3), 871–883 (2019) - 49.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res.

**15**(1), 1929–1958 (2014) - 50.
Kingma, D. P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint. arXiv:1412.6980 (2014)

- 51.
(2019) Keras documentation. [Online]. Available: https://keras.io

- 52.
(2019) Tensorflow. [Online]. Available: https://www.tensorflow.org

- 53.
Zhang, Y.: (2014) 6 months of abilene traffic matrices. [Online]. Available: http://www.cs.utexas.edu/~yzhang/

- 54.
Leduc, G., Abrahamsson, H., Balon, S., Bessler, S., D’Arienzo, M., Delcourt, O., Domingo-Pascual, J., Cerav-Erbas, S., Gojmerac, I., Masip, X., Pescapè, A., Quoitin, B., Romano, S., Salvadori, E., Skivée, F., Tran, H., Uhlig, S., Ümit, H.: An open source traffic engineering toolbox. Comput. Commun.

**29**(5), 593–610 (2006) networks of Excellence. http://www.sciencedirect.com/science/article/pii/S0140366405002124 - 55.
Leduc, G.: (2020) Toolbox for traffic engineering methods project (totem). [Online]. Available: https://totem.info.ucl.ac.be/index.html

- 56.
Howell, D.: (2004) Internet2. [Online]. Available: https://www.internet2.edu/news/detail/1978/

- 57.
Géron, A.: Hands-on Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O’Reilly Media, Inc. (2017)

- 58.
Yen, J .Y.: Finding the k shortest loopless paths in a network. Manage. Sci.

**17**(11), 712–716 (1971) - 59.
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.org

- 60.
Hinton, G., Srivastava, N., Swersky, K.: Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited on

**14**, 8 (2012) - 61.
Campbell, N.A., Atchley, W.R.: The geometry of canonical variate analysis. Syst. Zool.

**30**(3), 268–280 (1981). http://www.jstor.org/stable/2413249

## Acknowledgements

This work was supported and funded by Kuwait University Research Grant No. EO-07/18.

## Author information

### Affiliations

### Corresponding author

## Additional information

### Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Appendices

### Appendix 1: Column Generation-Based Approach

We give a brief description of the column generation-based approach presented in [16] to solve the MCNF with side constraints. The **CRP** is expressed in arc-flows rather than path-flows. Then, a subset of paths that satisfy the side constraints, i.e., feasible paths, is considered for each flow. Denote the set of feasible paths \(\bar{\mathcal {P}}_f = \{ p | p \in \mathcal {P}_f, \sum _{l \in \mathcal {L}} \delta _{pl} \omega _{fl} \le u_f\}, ~ \forall f \in \mathcal {F} \). Incorporating the side constraints into the set of feasible paths, allows formulating the problem as a linear program which is given by [16],

Here, \(\mathfrak {s}_f\) is an artificial variable associated with demand *f* constraint. Moreover, each variable has a cost coefficient \(\mathfrak {M}\). The theoretical difficulty of **CRP** is now hidden in \(\bar{\mathcal {P}}_f\), i.e., the generation of paths.

Let \(\mathfrak {\pi }_l\) and \(\kappa _f\), be dual variables associated with (17) and (18), respectively. Then, the **CRP-LP** problem can be decomposed into *F* *subproblems*, i.e., a problem for each flow, and the following column-generation-based approach is applied. In each iteration, the Procedure 1 provides upper and lower bounds. Therefore, it can be stopped when they are sufficiently close to each other.

A label correcting algorithm is used to solve each of the *subproblems* [16]. The algorithm allows creation of multiple labels, each with a cost and weight at each node. The labels are sorted in a priority queue in accordance with their cost value. In each iteration, a label is chosen and updated. Let *s*, *t*, *lim*, *Q* and *n*(*i*) be the source, the destination, the weight limit, the priority queue, and the number of labels at node *i*, respectively. Then, a formal description of the multi-labeling algorithm is given in Algorithm 1. Let \(\omega _{ij}\) be a weight coefficient of the link connecting nodes *i* and *j* for a given flow *f*; hence, it is equivalent to the a weight coefficient \(\omega _{fl}\). Similarly, let \(\bar{c}_{ij}\) be the link reduced costs corresponding to \(\bar{c}_{fl}\) of the link *l* connecting nodes *i* and *j*.

### Appendix 2: Complexity Analysis

In order to be able to compare the computational time of both MLMR framework and CGbA, the number of flows *F* can be written in terms of number of forwarding elements *N*. Assume that there are flows from each forwarding element to all other elements; hence, the number of flows \(F=N^2\). Let *h* be the number of layers in the proposed DNN. Then, the maximum number of neurons in any of the layers of the proposed DNN is \(KN^2\). Therefore, the runtime of *h* matrix multiplications is \(\mathcal {O}\left( h (KN^2)^3\right) \), i.e., \(\mathcal {O}(K^3 N^6)\), where \(K<< N\) and *h* is constant.

The CGbA consists of five major steps outlined in Procedure 1. The Floyd-Warshall algorithm can be used to identify the shortest feasible paths between every pair of nodes in first step. Then, the runtime of the first and second step is \(\mathcal {O}\left( N^3 \right) \). The **CRP-LP** is solved in step three for each flow \(f \in \mathcal {F}\). Given that the runtime of the label correcting algorithm is \(\mathcal {O}\left( lim^2N^2\right) \), where *lim* is bounded by *N* and \(F=N^2\), the third step runtime is \(\mathcal {O}(N^6)\) [16]. In step four and five, the maximum number of columns that can be added for all flows is \(N^2(K-1)\) and the runtime of solving the subproblem is \(\mathcal {O}( N^4)\). Thus, the runtime of steps three, four and five is \(\mathcal {O}\left( N^2(K-1)[N^6+N^4]\right) \). Hence, the overall time of CGbA is \(\mathcal {O}\left( N^8(K-1)\right) \).

The empirical and asymptotic computational analysis demonstrate superiority of the proposed MLMR framework to the CGbA.

## Rights and permissions

## About this article

### Cite this article

Awad, M.K., Ahmed, M.H.H., Almutairi, A.F. *et al.* Machine Learning-Based Multipath Routing for Software Defined Networks.
*J Netw Syst Manage* **29, **18 (2021). https://doi.org/10.1007/s10922-020-09583-4

Received:

Revised:

Accepted:

Published:

### Keywords

- Machine learning
- Software defined networks
- Software defined networking
- Routing