Peer4Peer: e-Science Community for Network Overlay and Grid Computing Research
This chapter describes a novel approach to Grid and overlay network research that leverages distributed infrastructures and multi-core machines enabling increased simulation complexity and speed. We present its motivation, background, current shortcomings, and the core architectural concepts of the novel research proposed. This is an ongoing effort to further our peer-to-peer cycle-sharing platform by providing a scalable, efficient, and reliable simulation substrate for the Grid and overlay topologies developed by the research community. Thus, Grid and overlay simulations are improved due to (1) increased scalability of simulation tools with a novel parallel, distributed, and decentralized architecture; (2) harnessing the power of idle CPU cycles spread around the Internet as a desktop Grid (over a peer-to-peer overlay); and (3) a framework for topology definition, dissemination, evaluation, and reuse which eases Grid and overlay research. The infrastructure, simulation engine, topology modeling language (TML), management services, and portal comprise a cloud-like platform for overlay research.
KeywordsVirtual Machine Resource Discovery Computing Infrastructure Grid Infrastructure Grid Simulation
The authors wish to thank students Filipe Paredes and João Neto for their implementation efforts. This work was supported by FCT (INESC-ID multiannual funding) through the PIDDAC Program funds.
- 1.H. Casanova. Simgrid: A Toolkit for the Simulation of Application Scheduling. First IEEE/ACM International Symposium on Cluster Computing and the Grid. 2001.Google Scholar
- 2.M. Jelasity, A. Montresor, Gian-Paolo Jesi, Spyros Voulgaris. The Peersim Simulator. http://peersim.sf.net.
- 3.I. Baumgart, B. Heep, S. Krause. OverSim: A Flexible Overlay Network Simulation Framework. In Proc. Of the IEEE Global Internet Symposium, pp. 79–84. 2007.Google Scholar
- 4.T. Issariyakul, E. Hossain Introduction to Network Simulator NS2. ISBN: 978-0-387-71759-3 Springer Books 2009.Google Scholar
- 5.R. Buyya, M. Murshed. GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing. Concurrency and Computation: Practice and Experience (CCPE), Volume 14, Issue 13–15. Wiley Press 2002.Google Scholar
- 6.V. S. Sunderam. PVM: a Framework for Parallel Distributed Computing. Concurrency: Practice and Experience, Volume 2, Issue 4, pp. 315–339. 1990.Google Scholar
- 7.T. E. Anderson, D. E. Culler, D. A. Patterson A Case for Networks of Workstations: NOW. IEEE Micro vol. 15-1, pp. 54–64. 1995.Google Scholar
- 8.D. P. Anderson, J. Cobb, E. Korpela, M. Lebofsky, D. Werthimer. SETI@home: An Experiment in Public-resource Computing. Commun. ACM 45, 11. November 2002.Google Scholar
- 9.Top 500 Supercomputers. http://www.top500.org. 2010.
- 10.S. Choi, H. Kim, E. Byun, M. Baik, S. Kim, C. Park, C. Hwang. Characterizing and Classifying Desktop Grid. Proceedings of the Seventh IEEE international Symposium on Cluster Computing and the Grid (May 14–17, 2007). 2007.Google Scholar
- 12.I. Foster. Designing and Building Parallel Programs (chapter 8 Message Passing Interface). ISBN 0201575949 Addison-Wesley. 1995.Google Scholar
- 13.M. Litzkow, M. Livny, M. Mutka, Condor: a Hunter of Idle Workstations. In Proceedings of the Eighth International Conference of Distributed Computing Systems, pp. 104–111. San Jose, CA, USA. 1988.Google Scholar
- 14.R. L. Henderson. Job Scheduling Under the Portable Batch System. In Proceedings of the Workshop on Job Scheduling Strategies For Parallel Processing. 1995.Google Scholar
- 15.W. Gentzsch. Sun Grid Engine: Towards Creating a Compute Power Grid. In Proceedings of the 1st international Symposium on Cluster Computing and the Grid. May 15–18. 2001.Google Scholar
- 17.Leach, P. and Perry, D.. Cifs: A common internet file system. Microsoft Interactive Developer. 1996.Google Scholar
- 18.F. Schmuck, R. Haskin. GPFS: A Shared-Disk File System for Large Computing Clusters. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, January 28–30, 2002.Google Scholar
- 19.Gluster Inc. GlusterFS. http://www.gluster.org. 2010.
- 20.D. G. Feitelson, L. Rudolph. Gang Scheduling Performance Benefits for Fine-Grain Synchronization. Journal of Parallel and Distributed Computing, volume 16, pages 306–318. 1992.Google Scholar
- 21.J.P. Jones, B. Nitzberg. Scheduling for Parallel Supercomputing: A Historical Perspective of Achievable Utilization. Job Scheduling Strategies for Parallel Processing (JSSPP), pp. 1–16, 1999.Google Scholar
- 23.J. K. Ousterhout. Scheduling Techniques for Concurrent Systems. Proceedings of Third International Conference on Distributed Computing Systems, 22–30. 1982.Google Scholar
- 24.Foster; C. Kesselman. The Grid: Blueprint for a New Computing Infrastructure. Morgan-Kaufman. 1999.Google Scholar
- 25.I. Foster. The Grid: A New Infrastructure for 21st Century Science. Physics Today, 55(2):42–47. 2002.Google Scholar
- 26.I. Foster. Globus Toolkit Version 4: Software for Service-Oriented Systems. IFIP International Conference on Network and Parallel Computing Springer-Verlag LNCS 3779, pp 2–13. 2006.Google Scholar
- 27.M. Feller, I. Foster, and S. Martin. GT4 GRAM: A Functionality and Performance Study. 2007.Google Scholar
- 28.Distributed Systems Architecture Group, Universidad Complutense de Madrid. GridWay Metascheduler http://www.gridway.org/. 2010.
- 29.J. Novotny, M. Russell, O. Wehrens, 2004. GridSphere: a Portal Framework for Building Collaborations: Research Articles. Concurr. Comput.: Pract. Exper. 16, 5: 503–513, Apr. 2004.Google Scholar
- 30.T. Suzumura, H. Nakada, M. Saito, S. Matsuoka, Y. Tanaka, S. Sekiguchi, The Ninf portal: an automatic generation tool for grid portals. In Proceedings of the 2002 Joint ACM-ISCOPE Conference on Java Grande (Seattle, Washington, USA, November 03–05, 2002). JGI ‘02. ACM, New York, NY, 2002.Google Scholar
- 31.D. Abramson, J. Giddy, L. Kotler. High Performance Parametric Modeling with Nimrod/G: Killer Application for the Global Grid?. Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS 2000). USA. 2000.Google Scholar
- 32.K. Harrison, W. T. L. P. Lavrijsen, C. E. Tull, P. Mato, A. Soroko, C. L. Tan, N. Brook, R. W. L. Jones. GANGA: a User-Grid Interface for Atlas and LHCb in Proceedings of Computing in High Energy and Nuclear Physics. La Jolla, 2003.Google Scholar
- 33.G. Klimeck, M. McLennan, S. P. Brophy, G. B. Adams III, M. S. Lundstrom. nanoHUB.org: Advancing Education and Research in Nanotechnology. Computing Science and Engg. 10, 5 (Sep. 2008), 17–23. 2008.Google Scholar
- 34.Plale, B., D. Gannon, J. Brotzge, K. Droegemeier, J. Kurose, D. McLaughlin, R. Wilhelmson, S. Graves, M. Ramamurthy, R.D. Clark, S. Yalda, D.A. Reed, E. Joseph, V. Chandrasekar, CASA and LEAD: Adaptive Cyberinfrastructure for Real-Time Multiscale Weather Forecasting, Computer (Special issue on System-Level Science), IEEE Computer Science Press, Vol. 39, No. 11, pp. 56–63, Nov. 2006.Google Scholar
- 36.D. P. Anderson and G. Fedak. The computational and storage potential of volunteer computing. In IEEE/ACM Intl. Symposium on Cluster Computing and the Grid. May 2006.Google Scholar
- 37.Amazon.com, Inc.: Amazon elastic compute cloud, http://aws.amazon.com/ec2. 2010.
- 38.Enomaly Inc. Enomalism: Elastic computing platform - virtual server management. http://enomalism.com.
- 39.OpenNebula Project Leads. OpenNebula, The Open Source Toolkit for Cloud Computing, http://www.opennebula.org/start. 2010.
- 40.C. Evangelinos, C. N. Hill. Cloud Computing for Parallel Scientific HPC Applications: Feasibility of Running Coupled Atmosphere-Ocean Climate Models on Amazon’s EC2. Proceedings of Cloud Computing and its Applications. http://www.cca08.org. 2008.
- 41.R. J. O. Figueiredo, P. O. Boykin, J. A. B. Fortes, T. Li, J. Peir, D. Wolinsky, L. K. John, D. R. Kaeli, D. J. Lilja, S. A. McKee, G. Memik, A. Roy, G. S. Tyson. Archer: A Community Distributed Computing Infrastructure for Computer Architecture Research and Education. 2008.Google Scholar
- 42.Brent N. Chun, David E. Culler, Timothy Roscoe, Andy C. Bavier, Larry L. Peterson, Mike Wawrzoniak, Mic Bowman: PlanetLab: an overlay test-bed for broad-coverage services. Computer Communication Review 33(3): 3–12. 2003.Google Scholar
- 43.Brian White, Jay Lepreau, Leigh Stoller, Robert Ricci, Shashi Guruprasad, Mac Newbold, Mike Hibler, Chad Barb, Abhijeet Joglekar. An Integrated Experimental Environment for Distributed Systems and Networks. OSDI 2002.Google Scholar
- 44.S. Naicken, B. Livingston, A. Basu, S. Rodhetbhai, Ian Wakeman, Dan Chalmers. The State of Peer-to-Peer Simulators and Simulations. Computer Communication Review, 2007.Google Scholar
- 45.Luís Veiga, André Negrão, Nuno Santos, Paulo Ferreira: Unifying divergence bounding and locality awareness in replicated systems with vector-field consistency. Journal of Internet Services and Applications vol. 1(2), pp. 95–115, 2010.Google Scholar
- 46.J. Silva, L. Veiga, P. Ferreira. nuBOINC: BOINC Extensions for Community Cycle Sharing. Workshop on Decentralized Self Management for grids, P2P, and User Communities - SELFMAN (in conjunction with SASo 2008). Venice, Italy. October 20–24, 2008.Google Scholar
- 47.J. N. Silva, L. Veiga, P. Ferreira. Service and Resource Discovery in Cycle Sharing Environments with an Utility Algebra 24th IEEE International Parallel & Distributed Processing Symposium, Atlanta (Georgia) USA. April 19–23, 2010.Google Scholar
- 48.L. Veiga, R. Rodrigues, P. Ferreira. GiGi: An Ocean of Gridlets on a “Grid-for-the-Masses”. Seventh IEEE International Symposium on Cluster Computing and the Grid, CCGRID 2007. Rio de Janeiro, Brazil. May 14–17, 2007.Google Scholar
- 49.J. Schopf, L. Pearlman, N. Miller, C. Kesselman, I. Foster, M. DArcy, and A. Chervenak, “Monitoring the grid with the Globus Toolkit MDS4,” in Journal of Physics: Conference Series, vol. 46, no. 1. Institute of Physics Publishing, 2006, pp. 521–525.Google Scholar
- 50.A. Rowstron, P. Druschel. Pastry: Scalable, Distributed Object Location and Routing for Large-scale Peer-to-Peer Systems. IFIP/ACM International Conference on Distributed Systems Platforms (Middleware).Vol. 11, pp. 329–350. 2001.Google Scholar
- 52.Terracotta, http://www.terracotta.org/, 2010
- 53.J. A. B. Fortes, R. J. Figueiredo, M. S. Lundstrom. Virtual Computing Infrastructures for Nanoelectronics Simulation Proceedings of the IEEE. 2005.Google Scholar
- 54.Ingmar Baumgart, Bernhard Heep, Stephan Krause. OverSim: A Flexible Overlay Network Simulation Framework. Proceedings of 10th IEEE Global Internet Symposium (GI ‘07) in conjunction with IEEE INFOCOM 2007. 2007.Google Scholar
- 55.H. Casanova, A. Legrand, M. Quinson. Simgrid: a Generic Framework for Large-Scale Distributed Experiments. 10th IEEE International Conference on Computer Modeling and Simulation. 2008.Google Scholar
- 56.Chunqiang Tang. DSF: A Common Platform for Distributed Systems Research and Development. In Proc. of Middleware 2009, pp. 414–436. 2009.Google Scholar
- 57.Marcel Dischinger, Andreas Haeberlen, Ivan Beschastnikh, P. Krishna Gummadi, Stefan Saroiu. Satellitelab: Adding Heterogeneity to Planetary-scale Network Test-beds. SIGCOMM 2008: 315–326. 2008.Google Scholar
- 58.F. Brasileiro, E. Araujo, W. Voorsluys, M. Oliveira, F. Figueiredo. Bridging the High Performance Computing Gap: the Ourgrid Experience. CCGRID 2007 Seventh IEEE International Symposium on Cluster Computing and the Grid. Rio de Janeiro, Brazil. 2007.Google Scholar
- 59.Yuanyuan Zhou, Liviu Iftode, Jaswinder Pal Singh, Kai Li, Brian R. Toonen, Ioannis Schoinas, Mark D. Hill, David A. Wood. Relaxed Consistency and Coherence Granularity in DSM Systems: A Performance Evaluation. Sixth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 1997.Google Scholar
- 60.Maurice Herlihy, Victor Luchangco, Mark Moir, and William N. Scherer III. Software Transactional Memory for Dynamic-Sized Data Structures. Proceedings of the Twenty-Second Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing 2003. 2003.Google Scholar
- 61.Terry, D.B., Theimer, M.M., Petersen, K., Demers, A.J., Spreitzer, M.J., Hauser, C.. Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System. Proceedings of the fifteenth ACM Symposium on Operating Systems Principles. 1995.Google Scholar
- 62.H. Yu, A. Vahdat. The Costs and Limits of Availability for Replicated Services. ACM Transactions on Computer Systems. 2006.Google Scholar
- 63.J. Chen, B. Wu, M. Delap, B. Knutsson, H. Lu, C. Amza. Locality Aware Dynamic Load Management for Massively Multiplayer Games. Proceedings of the tenth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 2005.Google Scholar
- 64.V. Lo, D. Zappala, D. Zhou, Y. Liu, and S. Zhao. Cluster Computing on the Fly: P2P Scheduling of Idle Cycles in the Internet. In The 3rd International Workshop on Peer-to-Peer Systems (IPTPS’04) 2004.Google Scholar
- 65.S. Androutsellis-Theotokis, D. Spinellis A Survey of Peer-to-Peer Content Distribution Technologies. ACM Computing Surveys, vol. 36–4, p. 371. 2004.Google Scholar
- 66.M. Castro, M. Costa, A. Rowstron. Debunking some Myths about Structured and Unstructured Overlays. Symposium on Networked Systems Design & Implementation. Volume 2, p. 98. 2005.Google Scholar
- 67.P. Trunfio, D. Talia, H. Papadakis, P. Fragopoulou, M. Mordacchini, M. Pennanen, K. Popov, V. Vlassov, S. Haridi. Peer-to-Peer Resource Discovery in Grids: Models and Systems. Future Generation Computer Systems, archive Volume 23. 2007.Google Scholar
- 68.M. Cai, M. Frank, J. Chen, P. Szekely. MAAN: A Multi-attribute Addressable Network for Grid Information Services. Proc. 4th Int. Workshop on Grid Computing, GRID 2003. 2003.Google Scholar