Adaptive Service Composition Based on Reinforcement Learning

  • Hongbing Wang
  • Xuan Zhou
  • Xiang Zhou
  • Weihong Liu
  • Wenya Li
  • Athman Bouguettaya
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6470)


The services on the Internet are evolving. The various properties of the services, such as their prices and performance, keep changing. To ensure user satisfaction in the long run, it is desirable that a service composition can automatically adapt to these changes. To this end, we propose a mechanism for adaptive service composition. The mechanism requires no prior knowledge about services’ quality, while being able to achieve the optimal composition solution by leveraging the technology of reinforcement learning. In addition, it allows a composite service to dynamically adjust itself to fit a varying environment, where the properties of the component services continue changing. We present the design of our mechanism, and demonstrate its effectiveness through an extensive experimental evaluation.


  1. 1.
    Agarwal, V., Dasgupta, K., Karnik, N.M., Kumar, A., Kundu, A., Mittal, S., Srivastava, B.: A service creation environment based on end to end composition of web services. In: WWW, pp. 128–137 (2005)Google Scholar
  2. 2.
    Ardagna, D., Pernici, B.: Global and local qos guarantee in web service selection. In: Bussler, C.J., Haller, A. (eds.) BPM 2005. LNCS, vol. 3812, pp. 32–46. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Berardi, D., Calvanese, D., Giacomo, G.D., Lenzerini, M., Mecella, M.: Automatic service composition based on behavioral descriptions. Int. J. Cooperative Inf. Syst. 14(4), 333–376 (2005)CrossRefGoogle Scholar
  4. 4.
    Chen, K., Xu, J., Reiff-Marganiec, S.: Markov-htn planning approach to enhance flexibility of automatic web service composition. In: ICWS, pp. 9–16 (2009)Google Scholar
  5. 5.
    Doshi, P., Goodwin, R., Akkiraju, R., Verma, K.: Dynamic workflow composition: Using markov decision processes. Int. J. Web Service Res. 2(1), 1–17 (2005)CrossRefGoogle Scholar
  6. 6.
    Gao, A., Yang, D., Tang, S., Zhang, M.: Web service composition using markov decision processes. In: Fan, W., Wu, Z., Yang, J. (eds.) WAIM 2005. LNCS, vol. 3739, pp. 308–319. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Hamadi, R., Benatallah, B.: A petri net-based model for web service composition. In: ADC, pp. 191–200 (2003)Google Scholar
  8. 8.
    Kaelbling, L.P., Littman, M.L., Moore, A.P.: Reinforcement learning: A survey. J. Artif. Intell. Res (JAIR) 4, 237–285 (1996)Google Scholar
  9. 9.
    Mei, L., Chan, W.K., Tse, T.H.: An adaptive service selection approach to service composition. In: ICWS, pp. 70–77 (2008)Google Scholar
  10. 10.
    Oh, M., Baik, J., Kang, S., Choi, H.-J.: An efficient approach for qos aware service selection based on a tree-based algorithm. In: ACIS-ICIS, pp. 605–610 (2008)Google Scholar
  11. 11.
    Oh, S.-C., Lee, D., Kumara, S.R.T.: Effective web service composition in diverse and large-scale service networks. IEEE TSC 1(1), 15–32 (2008)Google Scholar
  12. 12.
    Rao, J., Küngas, P., Matskin, M.: Composition of semantic web services using linear logic theorem proving. Inf. Syst. 31(4-5), 340–360 (2006)CrossRefGoogle Scholar
  13. 13.
    Rao, J., Su, X.: A survey of automated web service composition methods. In: Cardoso, J., Sheth, A.P. (eds.) SWSWPC 2004. LNCS, vol. 3387, pp. 43–54. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Shin, D.-H., Lee, K.-H., Suda, T.: Automated generation of composite web services based on functional semantics. J. Web Sem. 7(4), 332–343 (2009)CrossRefGoogle Scholar
  15. 15.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, Cambridge (1998)Google Scholar
  16. 16.
    Wang, Y.-L., Yu, X.-L.: Formalization and verification of automatic composition based on pi-calculus for semantic web service, December 1-30, vol. 1, pp. 103–106 (2009)Google Scholar
  17. 17.
    Watkins, C.J.C.H.: Learning from Delayed Rewards. PhD thesis, Kings College, Oxford (1989)Google Scholar
  18. 18.
    Yu, Q., Bouguettaya, A.: Framework for web service query algebra and optimization. TWEB 2(1) (2008)Google Scholar
  19. 19.
    Zeng, L., Benatallah, B., Ngu, A.H.H., Dumas, M., Kalagnanam, J., Chang, H.: Qos-aware middleware for web services composition. IEEE Trans. Softw. Eng. 30(5), 311–327 (2004)CrossRefGoogle Scholar
  20. 20.
    Zeng, L., Ngu, A., Benatallah, B., Podorozhny, R., Lei, H.: Dynamic composition and optimization of web services. Distributed and Parallel Databases 24(1), 45–72 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Hongbing Wang
    • 1
  • Xuan Zhou
    • 2
  • Xiang Zhou
    • 1
  • Weihong Liu
    • 1
  • Wenya Li
    • 1
  • Athman Bouguettaya
    • 2
  1. 1.School of Computer Science and EngineeringSoutheast UniversityChina
  2. 2.CSIRO ICT CentreAustralia

Personalised recommendations