Advertisement

Orlando Tools: Development, Training, and Use of Scalable Applications in Heterogeneous Distributed Computing Environments

  • Andrei TchernykhEmail author
  • Alexander Feoktistov
  • Sergei Gorsky
  • Ivan Sidorov
  • Roman Kostromin
  • Igor Bychkov
  • Olga Basharina
  • Vassil Alexandrov
  • Raul Rivera-Rodriguez
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 979)

Abstract

We address concepts and principles of the development, training, and use of applications in heterogeneous environments that integrate different computational infrastructures including HPC-clusters, grids, and clouds. Existing differences in the Grid and cloud computing models significantly complicate problem-solving processes in such environments for end-users. In this regards, we propose the toolkit named Orlando Tools for creating scalable applications for solving large-scale scientific and applied problems. It provides mechanisms for the subject domain specification, problem formulation, problem-solving time prediction, problem-solving scheme execution, monitoring, etc. The toolkit supports hands-on training skills for end-users. To demonstrate the practicability and benefits of Orlando Tools, we present an example of the development and use of the scalable application for solving practical problems of warehouse logistics.

Keywords

Scalable application Distributed computing HPC-cluster Grid Cloud Toolkit Training 

Notes

Acknowledgment

The study was partially supported by RFBR, projects no. 16-07-00931-a and no. 18-07-01224-a. Part of the work was supported by the Program of basic scientific research of the RAS, project no. IV.38.1.1.

References

  1. 1.
    Zhao, Y., Fei, Z., Raicu, I., Lu, S.: Opportunities and challenges in running scientific workflows on the cloud. In: Kumar, A., Xie, B., Lu, D. (eds.) Proceedings of the International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, pp. 455–462. IEEE, Piscataway (2011)Google Scholar
  2. 2.
    Zhan, Z.H., Liu, X.F., Gong, Y.J., Zhang, J., Chung, H.S.H., Li, Y.: Cloud computing resource scheduling and a survey of its evolutionary approaches. ACM Comput. Surv. 47(4), 1–33 (2015)CrossRefGoogle Scholar
  3. 3.
    Sokolinsky, L.B., Shamakina, A.V.: Methods of resource management in problem-oriented computing environment. Program. Comput. Softw. 42(1), 17–26 (2016)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Hollinsworth, D.: The workflow reference model. In: Zur Muehlen, M., Allen, R. (eds.) Workflow Management Coalition, Document no. TC00-1003 (1995)Google Scholar
  5. 5.
    Sowa, J.: Conceptual Structures – Information Processing in Mind and Machine. Addison-Wesley, Boston (1984)zbMATHGoogle Scholar
  6. 6.
    Tyugu, E.: Knowledge-Based Programming. Turing Institute Press, Glasgow (1988)zbMATHGoogle Scholar
  7. 7.
    Oracle Grid Engine. http://www.oracle.com/technetwork/oem/grid-engine-166852.html. Accessed 31 Jan 2018
  8. 8.
    Torque Resource Manager. http://www.adaptivecomputing.com/products/open-source/torque. Accessed 31 Jan 2018
  9. 9.
    HTCondor. http://research.cs.wisc.edu/htcondor. Accessed 31 Jan 2018
  10. 10.
    Slurm Workload Manager. http://slurm.net. Accessed 31 Jan 2018
  11. 11.
    GridWay Metascheduler. http://www.gridway. Accessed 31 Jan 2018
  12. 12.
    Frey, J., Tannenbaum, T., Livny, M.: Condor-G: a computation management agent for multi-institutional grids. Cluster Comput. 5(3), 237–246 (2002)CrossRefGoogle Scholar
  13. 13.
    Tao, J., Kolodziej, J., Ranjan, R., Jayaraman, P., Buyya, R.: A note on new trends in data-aware scheduling and resource provisioning in modern HPC systems. Future Gener. Commun. Syst. 51(C), 45–46 (2015)Google Scholar
  14. 14.
    Rings, T., et al.: Grid and cloud computing: opportunities for integration with the next generation network. J. Grid Comput. 7(3), Article no. 375 (2009)Google Scholar
  15. 15.
    Basili, V.R., et al.: Understanding the high-performance-computing community: a software engineer’s perspective. IEEE Softw. 25(4), 29–36 (2008)CrossRefGoogle Scholar
  16. 16.
    Joppa, L.N., et al.: Troubling trends in scientific software use. Science 340(6134), 814–815 (2013)CrossRefGoogle Scholar
  17. 17.
    Nunez, A., Merayo, M.G.: A formal framework to analyze cost and performance in map-reduce based applications. J. Comput. Sci. 5(2), 106–118 (2014)CrossRefGoogle Scholar
  18. 18.
    Pandey, S., Wu, L., Guru, S.M., Buyya, R.: A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments. In: 24th IEEE International Conference on Advanced Information Networking and Applications, pp. 400–407. IEEE (2010)Google Scholar
  19. 19.
    Yu, J., Buyya, R.: A taxonomy of workflow management systems for grid computing. J. Grid Comput. 3(3–4), 171–200 (2005)CrossRefGoogle Scholar
  20. 20.
    Barker, A., van Hemert, J.: Scientific workflow: a survey and research directions. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2007. LNCS, vol. 4967, pp. 746–753. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-68111-3_78CrossRefGoogle Scholar
  21. 21.
    Murugan, S., Kumar, S.: A survey of workflow management tools for grid platform. Adv. Inform. Technol. Manage. 1(1), 1–3 (2012)Google Scholar
  22. 22.
    Smirnov, P.A., Kovalchuk, S.V., Boukhanovsky, A.V.: Knowledge-based support for complex systems exploration in distributed problem solving environments. Commun. Comput. Inf. Sci. 394, 147–161 (2013)Google Scholar
  23. 23.
    Kliazovich, D., Pecero, J.E., Tchernykh, A., Bouvry, P., Khan, S.U., Zomaya, A.Y.: CA-DAG: modeling communication-aware applications for scheduling in cloud computing. J. Grid Comput. 14(1), 22–39 (2016)CrossRefGoogle Scholar
  24. 24.
    Talia, D.: Workflow systems for science: concepts and tools. ISRN Softw. Eng. 2013, 15 (2013). Article ID 404525CrossRefGoogle Scholar
  25. 25.
    Rodriguez, A., Tchernykh, A., Ecker, K.: Algorithms for dynamic scheduling of unit execution time tasks. Eur. J. Oper. Res. 146(2), 403–416 (2003)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Nesmachnow, S., Iturriaga, S., Dorronsoro, B., Tchernykh, A.: Multiobjective energy-aware workflow scheduling in distributed datacenters. Commun. Comput. Inf. Sci. 595, 79–93 (2016)Google Scholar
  27. 27.
    Cristobal, A., Tchernykh, A., Gaudiot, J.-L., Lin, W.-Y.: Non-strict execution in parallel and distributed computing. Int. J. Parallel Prog. 31(2), 77–105 (2003)CrossRefGoogle Scholar
  28. 28.
    Tchernykh, A., Schwiegelsohn, U., Alexandrov, V., Talbi, E.G.: Towards understanding uncertainty in cloud computing resource provisioning. Procedia Comput. Sci. 51, 1772–1781 (2015)CrossRefGoogle Scholar
  29. 29.
    Bychkov, I., Oparin, G., Tchernykh, A., Feoktistov, A., Bogdanova, V., Gorsky, S.: Conceptual model of problem-oriented heterogeneous distributed computing environment with multi-agent management. Procedia Comput. Sci. 103, 162–167 (2017)CrossRefGoogle Scholar
  30. 30.
    Intel® VTune™ Amplifier. https://software.intel.com/en-us/intel-vtune-amplifier-xe. Accessed 20 Apr 2018
  31. 31.
    Bychkov, I.V., Oparin, G.A., Feoktistov, A.G., Sidorov, I.A., Bogdanova, V.G., Gorsky, S.A.: Multiagent control of computational systems on the basis of meta-monitoring and imitational simulation. Optoelectron. Instrum. Data Process. 52(2), 107–112 (2016)CrossRefGoogle Scholar
  32. 32.
    Giarratano, J.C., Riley, G.D.: Expert Systems: Principles and Programming. Thomson, Boston (2005)Google Scholar
  33. 33.
    Feoktistov, A.G., Sidorov, I.A.: Logical-probabilistic analysis of distributed computing reliability. 39th International Convention on Information and Communication Technology, Electronics and Microelectronics, pp. 247–252. IEEE, Riejka (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Andrei Tchernykh
    • 1
    • 4
    Email author
  • Alexander Feoktistov
    • 2
  • Sergei Gorsky
    • 2
  • Ivan Sidorov
    • 2
  • Roman Kostromin
    • 2
  • Igor Bychkov
    • 2
  • Olga Basharina
    • 3
  • Vassil Alexandrov
    • 5
  • Raul Rivera-Rodriguez
    • 1
  1. 1.CICESE Research CenterEnsenadaMexico
  2. 2.Matrosov Institute for System Dynamics and Control Theory of SB RASIrkutskRussia
  3. 3.Irkutsk State UniversityIrkutskRussia
  4. 4.South Ural State UniversityChelyabinskRussia
  5. 5.ICREA-BSCBarcelonaSpain

Personalised recommendations