Abstract
Scientific Workflow Management Systems (SWfMS) are being used intensively to support large scale in silico experiments. In order to reduce execution time, current SWfMS have exploited workflow parallelization under the arising Many Tasks Computing (MTC) paradigm in homogeneous computing environments, such as multiprocessors, clusters and grids with centralized control. Although successful, this solution no longer applies to heterogeneous computing environments, such as hybrid clouds, which may combine users’ own computing resources with multiple edge clouds. A promising approach to address this challenge is Peer-to-Peer (P2P) which relies on decentralized control to deal with scalability and dynamic behavior of resources. In this paper, we propose a new P2P approach to apply MTC in scientific workflows. Through the results of simulation experiments, we show that our approach is promising.
Work partially sponsored by CNPq and INRIA within the Sarava équipe associée and by CAPES.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Deelman, E., Gannon, D., Shields, M., Taylor, I.: Workflows and e-Science: An overview of workflow system features and capabilities. Future Generation Computer Systems 25, 528–540 (2009)
Jarrard, R.D.: Scientific Methods (2001), http://emotionalcompetency.com/sci/booktoc.html
Raicu, I., Foster, I., Zhao, Y.: Many-task computing for grids and supercomputers. In: Workshop on Many-Task Computing on Grids and Supercomputers, pp. 1–11. Austin, Texas (2008)
Anderson, D.: BOINC: a system for public-resource computing and storage. In: Proceedings. Fifth IEEE/ACM International Workshop on Grid Computing, Pittsburgh, USA, pp. 4–10 (2004)
WCG, World Community Grid (2009), http://www.worldcommunitygrid.org
Grid4All Consortium, Towards Hybrid Clouds - A Grid4All perspective on cloud computing, White paper (2009), www.grid4all.eu
Freire, J., Koop, D., Santos, E., Silva, C.T.: Provenance for Computational Tasks: A Survey. Computing in Science and Engineering 10, 11–21 (2008)
Pacitti, E., Valduriez, P., Mattoso, M.: Grid Data Management: Open Problems and New Issues. Journal of Grid Computing 5, 273–281 (2007)
Lua, E.K., Crowcroft, J., Pias, M., Sharma, R., Lim, S.: A Survey and Comparison of Peer-to-Peer Overlay Network Schemes. IEEE Communications Surveys and Tutorials 7, 72–93 (2005)
Ganesan, P., Gummadi, K., Garcia-Molina, H.: Canon in G Major: Designing DHTs with Hierarchical Structure. In: Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS 2004), Tokyo, Japan, pp. 263–272 (2004)
Garcés-Erice, L., Biersack, E.W., Felber, P.A., Ross, K.W., Urvoy-Keller, G.: Hierarchical Peer-to-peer Systems. In: Proceedings of ACM/IFIP International Conference on Parallel and Distributed Computing, Klagenfurt, Austria, pp. 643–657 (2003)
Mastroianni, C., Talia, D., Verta, O.: A super-peer model for building resource discovery services in grids: Design and simulation analysis. In: Sloot, P.M.A., Hoekstra, A.G., Priol, T., Reinefeld, A., Bubak, M. (eds.) EGC 2005. LNCS, vol. 3470, pp. 132–143. Springer, Heidelberg (2005)
Martinez-Yelmo, I., Cuevas, R., Guerrero, C., Mauthe, A.: Routing Performance in a Hierarchical DHT-based Overlay Network. In: Euromicro Conference on Parallel, Distributed, and Network-Based Processing, pp. 508–515. IEEE Computer Society Press, Los Alamitos (2008)
Ogasawara, E., Oliveira, D., Chirigati, F., Barbosa, C.E., Elias, R., Braganholo, V., Coutinho, A., Mattoso, M.: Exploring many task computing in scientific workflows. In: MTAGS 2009, pp. 1–10. ACM Press, Portland (2009)
Dick, M.E., Pacitti, E., Kemme, B.: Flower-CDN: a hybrid P2P overlay for efficient query processing in CDN. In: Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology, pp. 427–438. ACM, Saint Petersburg (2009)
Akbarinia, R., Pacitti, E., Valduriez, P.: Data currency in replicated DHTs. In: Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, pp. 211–222. ACM, Beijing (2007)
Karakaya, M., Körpeoğlu, İ., Ulusoy, Ö.: A connection management protocol for promoting cooperation in Peer-to-Peer networks. Comput. Commun. 31, 240–256 (2008)
Almeida, E.C.D., Sunyé, G., Traon, Y.L., Valduriez, P.: A Framework for Testing Peer-to-Peer Systems. In: Proceedings of the 2008 19th International Symposium on Software Reliability Engineering, Los Alamitos, CA, USA, pp. 167–176 (2008)
Jelasity, M., Montresor, A., Jesi, G.P., Voulgaris, S.: The PeerSim simulator (2010), http://peersim.sourceforge.net
Freedman, D., Pisani, R., Purves, R.: Statistics, 4th edn. W. W. Norton, New York (2007)
Law, A.M.: Statistical analysis of simulation output data: the practical state of the art. In: Proceedings of the 39th Conference on Winter Simulation, pp. 77–83. IEEE Press, Washington D.C (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ogasawara, E. et al. (2011). A P2P Approach to Many Tasks Computing for Scientific Workflows. In: Palma, J.M.L.M., Daydé, M., Marques, O., Lopes, J.C. (eds) High Performance Computing for Computational Science – VECPAR 2010. VECPAR 2010. Lecture Notes in Computer Science, vol 6449. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19328-6_31
Download citation
DOI: https://doi.org/10.1007/978-3-642-19328-6_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19327-9
Online ISBN: 978-3-642-19328-6
eBook Packages: Computer ScienceComputer Science (R0)