Abstract
Quality of service (QoS) is gaining more and more importance in today’s networks. We present a fully decentralized and self-organizing approach for QoS routing and Traffic Engineering in connection oriented networks, e.g. MPLS networks. Based on reinforcement learning the algorithm learns the optimal routing policy for incoming connection requests while minimizing the blocking probability. In contrast to other approaches our method does not rely on predefined paths or LSPs and is able to optimize the network utilization in the presence of multiple QoS restrictions like bandwidth and delay. Moreover, no additional signaling overhead is required. Using an adaptive neural vector quantization technique for clustering the state space a considerable speed-up of learning the routing policy can be achieved. In different experiments we are able to show that our approach performs better than classical approaches like Widest Shortest Path routing (WSP).
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Rosen, E., Viswanathan, A., Callon, R.: Multiprotocol Label Switching Architecture. IETF RFC 3031 (2001)
Evans, J., Filsfils, C.: Deploying IP and MPLS QoS for Multiservice Networks: Theory and Practice. Morgan Kaufmann, San Francisco (2007)
Liu, Y., Tham, C., Hui, T.: MAPS: A Localized and Distributed Adaptive Path Selection Scheme in MPLS Networks. In: Proc. of IEEE Workshop on High Performance Switching and Routing (HPSR) (2003)
Kodialam, M.S., Lakshman, T.V.: Minimum Interference Routing with Applications to MPLS Traffic Engineering. In: INFOCOM, vol. (2), pp. 884–893 (2000)
Awduche, D., Malcolm, J., Agogbua, J., ODell, M., Mcmanus, J.: Requirements for Traffic Engineering Over MPLS. IETF RFC 2702 (1999)
Guerin, R., Williams, D., Orda, A.: QoS Routing Mechanisms and OSPF Extensions. In: Proc. of Globecom (1997)
Figueiredo, G., da Fonseca, N., Monteiro, J.A.S.: A minimum interference routing algorithm. In: Proc. of the IEEE Int. Conf. on Communications, vol. 4 (2004)
Boutaba, R., Szeto, W., Iraqi, Y.: DORA: Efficient Routing for MPLS Traffic Engineering. Journal of Network and Systems Management, Special Issue on Internet Traffic Engineering and Management 10, 309–325 (2002)
Carrillo, L., Marzo, J., Vilà, P., Fàbrega, L., Guadall, C.: A Quality of Service Routing Scheme for Packet Switched Networks based on Ant Colony Behavior. In: Proc. of the Int. Symposium on Performance Evaluation of Computer and Telecommunication Systems, pp. 637–641 (2004)
Boyan, J.A., Littman, M.L.: Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach. In: Advances in Neural Information Processing Systems, vol. 6, pp. 671–678. Morgan Kaufmann Publishers, Inc., San Francisco (1994)
Peshkin, L., Savova, V.: Reinforcement learning for adaptive routing. In: Proc. of the International Joint Conference on Neural Networks (IJCNN) (2002)
Heidari, F., Mannor, S., Mason, L.: Reinforcement learning-based load shared sequential routing. In: Proc. of the IFIP Networking (2007)
Rummery, G.A., Niranjan, M.: On-line Q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR 166, Cambridge University Engineering Department (1994)
Watkins, C.: Learning from Delayed Rewards. PhD thesis, King’s College, Cambridge University, UK (1989)
Sutton, R., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., Swallow, G.: RSVP-TE: Extensions to RSVP for LSP Tunnels. IETF RFC 3209 (2001)
Gross, H.M., Stephan, V., Krabbes, M.: A Neural Field Approach to Topological Reinforcement Learning in Continous Action Spaces. In: Proc. 1998 IEEE World Congress on Computational Intelligence WCCI 1998, pp. 1992–1997 (1998)
Fritzke, B.: A Growing Neural Gas Network Learns Topologies. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems 7, pp. 625–632. MIT Press, Cambridge (1995)
Marsland, S., Shapiro, J., Nehmzow, U.: A self-organising network that grows when required. Neural Networks 15, 1041–1058 (2002)
Watteyne, T., Auge-Blum, I., Dohler, M., Barthel, D.: Geographic Forwarding in Wireless Sensor Networks with Loose Position-Awareness. In: Personal, Indoor and Mobile Radio Communications, PIMRC, pp. 1–5 (2007)
Navas, J.C., Imielinski, T.: GeoCast – Geographic Addressing and Routing. In: Mobile Computing and Networking, pp. 66–76 (1997)
Andoni, A., Indyk, P.: Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions. In: 47th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2006, pp. 459–468 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 IFIP International Federation for Information Processing
About this paper
Cite this paper
Einhorn, E., Mitschele-Thiel, A. (2008). RLTE: Reinforcement Learning for Traffic-Engineering. In: Hausheer, D., Schönwälder, J. (eds) Resilient Networks and Services. AIMS 2008. Lecture Notes in Computer Science, vol 5127. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70587-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-70587-1_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70586-4
Online ISBN: 978-3-540-70587-1
eBook Packages: Computer ScienceComputer Science (R0)