A Novel Approach to Large-Scale Services Composition

  • Hongbing Wang
  • Xiaojun Wang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7808)


We investigate a multi-agent reinforcement learning model for the optimization of Web service composition in this paper. Based on the model, a multi-agent Q-learning algorithm was proposed, where agents in a team would benefit from one another. In contrast to single-agent reinforcement-learning, our algorithm can speed up the convergence to optimal policy. In addition, it allows composite service to dynamically adjust itself to fit a varying environment, where the properties of the component services continue changing. A set of experiments is given to prove the efficiency of the analysis. The advantages and the limitations of the proposed approach are also discussed.


Web Service composition multi-agent 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Busoniu, L., Babuska, R., De Schutter, B.: A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 38(2), 156–172 (2008)CrossRefGoogle Scholar
  2. 2.
    Carman, M., Serafini, L., Traverso, P.: Web service composition as planning. In: ICAPS 2003 Workshop on Planning for Web Services, pp. 1636–1642 (2003)Google Scholar
  3. 3.
    Doshi, P., Goodwin, R., Akkiraju, R., Verma, K.: Dynamic workflow composition using markov decision processes. In: IEEE International Conference on Web Services, pp. 576–582. IEEE (2004)Google Scholar
  4. 4.
    Gao, A., Yang, D., Tang, S., Zhang, M.: Web service composition using markov decision processes. In: Fan, W., Wu, Z., Yang, J. (eds.) WAIM 2005. LNCS, vol. 3739, pp. 308–319. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Gonzaga, T., Bentes, C., Farias, R., de Castro, M., Garcia, A.: Using distributed-shared memory mechanisms for agents communication in a distributed system. In: Seventh International Conference on Intelligent Systems Design and Applications, ISDA 2007, pp. 39–46. IEEE (2007)Google Scholar
  6. 6.
    Hwang, S.Y., Lim, E.P., Lee, C.H., Chen, C.H.: Dynamic web service selection for reliable web service composition. IEEE Transactions on Services Computing 1(2), 104–116 (2008)CrossRefGoogle Scholar
  7. 7.
    Kaelbling, L., Littman, M., Moore, A.: Reinforcement learning: A survey. Arxiv preprint cs/9605103 (1996)Google Scholar
  8. 8.
    Papazoglou, M., Georgakopoulos, D.: Service-oriented computing. Communications of the ACM 46(10), 25–28 (2003)CrossRefGoogle Scholar
  9. 9.
    Sirin, E., Parsia, B., Wu, D., Hendler, J., Nau, D.: Htn planning for web service composition using shop2. Web Semantics: Science, Services and Agents on the World Wide Web 1(4), 377–396 (2004)CrossRefGoogle Scholar
  10. 10.
    Sutton, R., Barto, A.: Reinforcement learning. Journal of Cognitive Neuroscience 11(1), 126–134 (1999)CrossRefGoogle Scholar
  11. 11.
    Wang, H., Zhou, X., Zhou, X., Liu, W., Li, W., Bouguettaya, A.: Adaptive service composition based on reinforcement learning. In: Maglio, P.P., Weske, M., Yang, J., Fantinato, M. (eds.) ICSOC 2010. LNCS, vol. 6470, pp. 92–107. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Hongbing Wang
    • 1
  • Xiaojun Wang
    • 1
  1. 1.School of Computer Science and EngineeringSoutheast UniversityNanjingChina

Personalised recommendations