Advertisement

Usability of Scientific Workflow in Dynamically Changing Environment

  • Anna BánátiEmail author
  • Eszter Kail
  • Péter Kacsuk
  • Miklos Kozlovszky
Conference paper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 450)

Abstract

Scientific workflow management systems are mainly data-flow oriented, which face several challenges due to the huge amount of data and the required computational capacity which cannot be predicted before enactment. Other problems may arise due to the dynamic access of the data storages or other data sources and the distributed nature of the scientific workflow computational infrastructures (cloud, cluster, grid, HPC), which status may change even during running of a single workflow instance. Many of these failures could be avoided with workflow management systems that provide provenance based dynamism and adaptivity to the unforeseen scenarios arising during enactment. In our work we summarize and categorize the failures that can arise in cloud environment during enactment and show the possibility of prediction and avoidance of failures with dynamic and provenance support.

Keywords

Scientific workflow Dynamic workflow management system Distributed computing Cloud failures Fault tolerance 

References

  1. 1.
    da Cruz, S.M.S., Paulino, C.E., de Oliveira, D., Campos, M.L.M., Mattoso, M.: Capturing distributed provenance metadata from cloud-based scientific workflows. Journal of Information and Data Management 2(1), 43 (2011)Google Scholar
  2. 2.
    Samak, T., Gunter, D., Goode, M., Deelman, E., Juve, G., Silva, F., Vahi, K.: Failure analysis of distributed scientific workflows executing in the cloud. In: 2012 8th International Conference on Network and Service Management (CNSM) and 2012 Workshop on Systems Virtualiztion Management (SVM), pp. 46–54. IEEE (2012)Google Scholar
  3. 3.
    Liang, Y., Zhang, Y., Jette, M., Sivasubramaniam, A., Sahoo, R.: BlueGene/L failure analysis and prediction models. In: International Conference on Dependable Systems and Networks (DSN), pp. 425–434 (2006)Google Scholar
  4. 4.
    Pham, C., Cao, P., Kalbarczyk, Z., Iyer, R.K.: Toward a high availability cloud: Techniques and challenges. In: IEEE/IFIP 42nd International Conference on Dependable Systems and Networks Workshops (DSN-W), pp. 1–6. IEEE (2012)Google Scholar
  5. 5.
    Chen, X., Lu, C. D., Pattabiraman, K.: Failure analysis of jobs in compute clouds: A google cluster case study. In: The International Symposium on Software Reliability Engineering (ISSRE). IEEE (2014)Google Scholar
  6. 6.
    Vishwanath, K.W., Nachiappan, N.: Characterizing cloud computing hardware reliability. In: 1st ACM Symposium on Cloud Computing. ACM (2010)Google Scholar
  7. 7.
    Bala, A., Chana, I.: Fault tolerance-challenges, techniques and implementation in cloud computing. IJCSI International Journal of Computer Science Issues 9(1) (2012) ISSN (Online): 1694-0814Google Scholar
  8. 8.
    Plankensteiner, K., Prodan, R, Fahringer, T., Kertesz, A., Kacsuk, P.: Fault-tolerant behavior in state-of-the-art grid workflow management systems (2007)Google Scholar
  9. 9.
    Bahsi, E.M.: Dynamic Workflow Management For Large Scale Scientific Applications, PhD Thesis, B.S., Fatih University (2006)Google Scholar
  10. 10.
    Deelman, E., Gil, Y., Ellisman, M., Fahringer, T., Fox, G., Gannon, D., Goble, C., Livny, M., Moreau, L., Myers, J.: Examining the challenges of scientific workflows. IEEE Computer 40(12), 26–34 (2007)Google Scholar
  11. 11.
    Deelman, E., Gil, Y.: Managing large-scale scientific workflows in distributed environments: Experiences and challenges. In: e-Science, p. 144 (2006)Google Scholar
  12. 12.
    Ludäscher, B., Altintas, I., Bowers, S., Cummings, J., Critchlow, T., Deelman, E., Vouk, M.: Scientific process automation and workflow management. In: Scientific Data Management: Challenges, Existing Technology, and Deployment. Computational Science Series, pp. 476–508 (2009)Google Scholar
  13. 13.
    Kail, E., Bánáti, A., Karóczkai, K., Kacsuk, P., Kozlovszky, M.: Dynamic workflow support in gUSE. In: Proceedings of the 37th International Convention, MIPRO (2014)Google Scholar
  14. 14.
    Kail, E., Bánáti, A., Kacsuk, P., Kozlovszky, M.: Provenance based adaptive and dynamic workflows. In: 15th IEEE International Symposium on Computational Intelligence and Informatics, pp 215–219. IEEE Press, Budapest (2014)Google Scholar
  15. 15.
    Das, A.: On Fault Tolerance of Resources in Computational Grids. International Journal of Grid Computing & Applications 3, 1–10 (2012)CrossRefGoogle Scholar
  16. 16.
    Mouallem, P.A., Vouk, M.: A fault tolerance framework for kepler-based distributed scientific workflows. North Carolina State University (2011)Google Scholar
  17. 17.
    Alsoghayer, R.A.: Risk assessment models for resource failure in grid computing. Thesis, University of Leeds (2011)Google Scholar
  18. 18.
    Bánáti, A., Kacsuk, P., Kozlovszky, M.: Towards flexible provenance and workflow manipulation in scientific workflows. In: Proceedings of CGW 2014 (2014)Google Scholar
  19. 19.
    Kail, E., Kacsuk, P., Kozlovszky, M.: A novel approach to user-steering in scientific workflow. In: Proceedings of CGW 2014 (2014)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  • Anna Bánáti
    • 1
    Email author
  • Eszter Kail
    • 1
  • Péter Kacsuk
    • 2
    • 3
  • Miklos Kozlovszky
    • 1
    • 2
  1. 1.John von Neumann Faculty of Informatics, Biotech LabObuda UniversityBudapestHungary
  2. 2.LPDSMTA SZTAKIBudapestHungary
  3. 3.University of WestminsterLondonUK

Personalised recommendations