Advertisement

Peer4Peer: e-Science Community for Network Overlay and Grid Computing Research

  • Luís Veiga
  • João Nuno Silva
  • João Coelho Garcia
Chapter
Part of the Computer Communications and Networks book series (CCN)

Abstract

This chapter describes a novel approach to Grid and overlay network research that leverages distributed infrastructures and multi-core machines enabling increased simulation complexity and speed. We present its motivation, background, current shortcomings, and the core architectural concepts of the novel research proposed. This is an ongoing effort to further our peer-to-peer cycle-sharing platform by providing a scalable, efficient, and reliable simulation substrate for the Grid and overlay topologies developed by the research community. Thus, Grid and overlay simulations are improved due to (1) increased scalability of simulation tools with a novel parallel, distributed, and decentralized architecture; (2) harnessing the power of idle CPU cycles spread around the Internet as a desktop Grid (over a peer-to-peer overlay); and (3) a framework for topology definition, dissemination, evaluation, and reuse which eases Grid and overlay research. The infrastructure, simulation engine, topology modeling language (TML), management services, and portal comprise a cloud-like platform for overlay research.

Keywords

Virtual Machine Resource Discovery Computing Infrastructure Grid Infrastructure Grid Simulation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

The authors wish to thank students Filipe Paredes and João Neto for their implementation efforts. This work was supported by FCT (INESC-ID multiannual funding) through the PIDDAC Program funds.

References

  1. 1.
    H. Casanova. Simgrid: A Toolkit for the Simulation of Application Scheduling. First IEEE/ACM International Symposium on Cluster Computing and the Grid. 2001.Google Scholar
  2. 2.
    M. Jelasity, A. Montresor, Gian-Paolo Jesi, Spyros Voulgaris. The Peersim Simulator. http://peersim.sf.net.
  3. 3.
    I. Baumgart, B. Heep, S. Krause. OverSim: A Flexible Overlay Network Simulation Framework. In Proc. Of the IEEE Global Internet Symposium, pp. 79–84. 2007.Google Scholar
  4. 4.
    T. Issariyakul, E. Hossain Introduction to Network Simulator NS2. ISBN: 978-0-387-71759-3 Springer Books 2009.Google Scholar
  5. 5.
    R. Buyya, M. Murshed. GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing. Concurrency and Computation: Practice and Experience (CCPE), Volume 14, Issue 13–15. Wiley Press 2002.Google Scholar
  6. 6.
    V. S. Sunderam. PVM: a Framework for Parallel Distributed Computing. Concurrency: Practice and Experience, Volume 2, Issue 4, pp. 315–339. 1990.Google Scholar
  7. 7.
    T. E. Anderson, D. E. Culler, D. A. Patterson A Case for Networks of Workstations: NOW. IEEE Micro vol. 15-1, pp. 54–64. 1995.Google Scholar
  8. 8.
    D. P. Anderson, J. Cobb, E. Korpela, M. Lebofsky, D. Werthimer. SETI@home: An Experiment in Public-resource Computing. Commun. ACM 45, 11. November 2002.Google Scholar
  9. 9.
    Top 500 Supercomputers. http://www.top500.org. 2010.
  10. 10.
    S. Choi, H. Kim, E. Byun, M. Baik, S. Kim, C. Park, C. Hwang. Characterizing and Classifying Desktop Grid. Proceedings of the Seventh IEEE international Symposium on Cluster Computing and the Grid (May 14–17, 2007). 2007.Google Scholar
  11. 11.
    B. Schmidt. A Survey of Desktop Grid Applications for E-science. Int. J. Web Grid Serv. 3, 3 (Aug. 2007), 354–368. 2007.CrossRefGoogle Scholar
  12. 12.
    I. Foster. Designing and Building Parallel Programs (chapter 8 Message Passing Interface). ISBN 0201575949 Addison-Wesley. 1995.Google Scholar
  13. 13.
    M. Litzkow, M. Livny, M. Mutka, Condor: a Hunter of Idle Workstations. In Proceedings of the Eighth International Conference of Distributed Computing Systems, pp. 104–111. San Jose, CA, USA. 1988.Google Scholar
  14. 14.
    R. L. Henderson. Job Scheduling Under the Portable Batch System. In Proceedings of the Workshop on Job Scheduling Strategies For Parallel Processing. 1995.Google Scholar
  15. 15.
    W. Gentzsch. Sun Grid Engine: Towards Creating a Compute Power Grid. In Proceedings of the 1st international Symposium on Cluster Computing and the Grid. May 15–18. 2001.Google Scholar
  16. 16.
    Howard, J.H., Kazar, M.L., Nichols, S.G., Nichols, D.A., Satyanarayanan, M., Sidebotham, R.N., & West, M.J.. Scale and Performance in a Distributed File System. ACM Transactions on Computer Systems 6 (1): 51–81. February 1988.CrossRefGoogle Scholar
  17. 17.
    Leach, P. and Perry, D.. Cifs: A common internet file system. Microsoft Interactive Developer. 1996.Google Scholar
  18. 18.
    F. Schmuck, R. Haskin. GPFS: A Shared-Disk File System for Large Computing Clusters. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, January 28–30, 2002.Google Scholar
  19. 19.
    Gluster Inc. GlusterFS. http://www.gluster.org. 2010.
  20. 20.
    D. G. Feitelson, L. Rudolph. Gang Scheduling Performance Benefits for Fine-Grain Synchronization. Journal of Parallel and Distributed Computing, volume 16, pages 306–318. 1992.Google Scholar
  21. 21.
    J.P. Jones, B. Nitzberg. Scheduling for Parallel Supercomputing: A Historical Perspective of Achievable Utilization. Job Scheduling Strategies for Parallel Processing (JSSPP), pp. 1–16, 1999.Google Scholar
  22. 22.
    Y. K. Kwok and I. Ahmad. Static Scheduling Algorithms for Allocating Directed Task Graphs to Multiprocessors. ACM Comput. Surv., 31(4):406–471, 1999.CrossRefGoogle Scholar
  23. 23.
    J. K. Ousterhout. Scheduling Techniques for Concurrent Systems. Proceedings of Third International Conference on Distributed Computing Systems, 22–30. 1982.Google Scholar
  24. 24.
    Foster; C. Kesselman. The Grid: Blueprint for a New Computing Infrastructure. Morgan-Kaufman. 1999.Google Scholar
  25. 25.
    I. Foster. The Grid: A New Infrastructure for 21st Century Science. Physics Today, 55(2):42–47. 2002.Google Scholar
  26. 26.
    I. Foster. Globus Toolkit Version 4: Software for Service-Oriented Systems. IFIP International Conference on Network and Parallel Computing Springer-Verlag LNCS 3779, pp 2–13. 2006.Google Scholar
  27. 27.
    M. Feller, I. Foster, and S. Martin. GT4 GRAM: A Functionality and Performance Study. 2007.Google Scholar
  28. 28.
    Distributed Systems Architecture Group, Universidad Complutense de Madrid. GridWay Metascheduler http://www.gridway.org/. 2010.
  29. 29.
    J. Novotny, M. Russell, O. Wehrens, 2004. GridSphere: a Portal Framework for Building Collaborations: Research Articles. Concurr. Comput.: Pract. Exper. 16, 5: 503–513, Apr. 2004.Google Scholar
  30. 30.
    T. Suzumura, H. Nakada, M. Saito, S. Matsuoka, Y. Tanaka, S. Sekiguchi, The Ninf portal: an automatic generation tool for grid portals. In Proceedings of the 2002 Joint ACM-ISCOPE Conference on Java Grande (Seattle, Washington, USA, November 03–05, 2002). JGI ‘02. ACM, New York, NY, 2002.Google Scholar
  31. 31.
    D. Abramson, J. Giddy, L. Kotler. High Performance Parametric Modeling with Nimrod/G: Killer Application for the Global Grid?. Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS 2000). USA. 2000.Google Scholar
  32. 32.
    K. Harrison, W. T. L. P. Lavrijsen, C. E. Tull, P. Mato, A. Soroko, C. L. Tan, N. Brook, R. W. L. Jones. GANGA: a User-Grid Interface for Atlas and LHCb in Proceedings of Computing in High Energy and Nuclear Physics. La Jolla, 2003.Google Scholar
  33. 33.
    G. Klimeck, M. McLennan, S. P. Brophy, G. B. Adams III, M. S. Lundstrom. nanoHUB.org: Advancing Education and Research in Nanotechnology. Computing Science and Engg. 10, 5 (Sep. 2008), 17–23. 2008.Google Scholar
  34. 34.
    Plale, B., D. Gannon, J. Brotzge, K. Droegemeier, J. Kurose, D. McLaughlin, R. Wilhelmson, S. Graves, M. Ramamurthy, R.D. Clark, S. Yalda, D.A. Reed, E. Joseph, V. Chandrasekar, CASA and LEAD: Adaptive Cyberinfrastructure for Real-Time Multiscale Weather Forecasting, Computer (Special issue on System-Level Science), IEEE Computer Science Press, Vol. 39, No. 11, pp. 56–63, Nov. 2006.Google Scholar
  35. 35.
    Ian J. Taylor, Ian Wang, Matthew S. Shields, Shalil Majithia: Distributed computing with Triana on the Grid. Concurrency and Computation: Practice and Experience 17(9): 1197–1214, 2005.CrossRefGoogle Scholar
  36. 36.
    D. P. Anderson and G. Fedak. The computational and storage potential of volunteer computing. In IEEE/ACM Intl. Symposium on Cluster Computing and the Grid. May 2006.Google Scholar
  37. 37.
    Amazon.com, Inc.: Amazon elastic compute cloud, http://aws.amazon.com/ec2. 2010.
  38. 38.
    Enomaly Inc. Enomalism: Elastic computing platform - virtual server management. http://enomalism.com.
  39. 39.
    OpenNebula Project Leads. OpenNebula, The Open Source Toolkit for Cloud Computing, http://www.opennebula.org/start. 2010.
  40. 40.
    C. Evangelinos, C. N. Hill. Cloud Computing for Parallel Scientific HPC Applications: Feasibility of Running Coupled Atmosphere-Ocean Climate Models on Amazon’s EC2. Proceedings of Cloud Computing and its Applications. http://www.cca08.org. 2008.
  41. 41.
    R. J. O. Figueiredo, P. O. Boykin, J. A. B. Fortes, T. Li, J. Peir, D. Wolinsky, L. K. John, D. R. Kaeli, D. J. Lilja, S. A. McKee, G. Memik, A. Roy, G. S. Tyson. Archer: A Community Distributed Computing Infrastructure for Computer Architecture Research and Education. 2008.Google Scholar
  42. 42.
    Brent N. Chun, David E. Culler, Timothy Roscoe, Andy C. Bavier, Larry L. Peterson, Mike Wawrzoniak, Mic Bowman: PlanetLab: an overlay test-bed for broad-coverage services. Computer Communication Review 33(3): 3–12. 2003.Google Scholar
  43. 43.
    Brian White, Jay Lepreau, Leigh Stoller, Robert Ricci, Shashi Guruprasad, Mac Newbold, Mike Hibler, Chad Barb, Abhijeet Joglekar. An Integrated Experimental Environment for Distributed Systems and Networks. OSDI 2002.Google Scholar
  44. 44.
    S. Naicken, B. Livingston, A. Basu, S. Rodhetbhai, Ian Wakeman, Dan Chalmers. The State of Peer-to-Peer Simulators and Simulations. Computer Communication Review, 2007.Google Scholar
  45. 45.
    Luís Veiga, André Negrão, Nuno Santos, Paulo Ferreira: Unifying divergence bounding and locality awareness in replicated systems with vector-field consistency. Journal of Internet Services and Applications vol. 1(2), pp. 95–115, 2010.Google Scholar
  46. 46.
    J. Silva, L. Veiga, P. Ferreira. nuBOINC: BOINC Extensions for Community Cycle Sharing. Workshop on Decentralized Self Management for grids, P2P, and User Communities - SELFMAN (in conjunction with SASo 2008). Venice, Italy. October 20–24, 2008.Google Scholar
  47. 47.
    J. N. Silva, L. Veiga, P. Ferreira. Service and Resource Discovery in Cycle Sharing Environments with an Utility Algebra 24th IEEE International Parallel & Distributed Processing Symposium, Atlanta (Georgia) USA. April 19–23, 2010.Google Scholar
  48. 48.
    L. Veiga, R. Rodrigues, P. Ferreira. GiGi: An Ocean of Gridlets on a “Grid-for-the-Masses”. Seventh IEEE International Symposium on Cluster Computing and the Grid, CCGRID 2007. Rio de Janeiro, Brazil. May 14–17, 2007.Google Scholar
  49. 49.
    J. Schopf, L. Pearlman, N. Miller, C. Kesselman, I. Foster, M. DArcy, and A. Chervenak, “Monitoring the grid with the Globus Toolkit MDS4,” in Journal of Physics: Conference Series, vol. 46, no. 1. Institute of Physics Publishing, 2006, pp. 521–525.Google Scholar
  50. 50.
    A. Rowstron, P. Druschel. Pastry: Scalable, Distributed Object Location and Routing for Large-scale Peer-to-Peer Systems. IFIP/ACM International Conference on Distributed Systems Platforms (Middleware).Vol. 11, pp. 329–350. 2001.Google Scholar
  51. 51.
    A. Rowstron e P. Druschel, “Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility,” ACM SIGOPS Operating Systems Review, vol. 35, 2001, pp. 188–201.CrossRefGoogle Scholar
  52. 52.
    Terracotta, http://www.terracotta.org/, 2010
  53. 53.
    J. A. B. Fortes, R. J. Figueiredo, M. S. Lundstrom. Virtual Computing Infrastructures for Nanoelectronics Simulation Proceedings of the IEEE. 2005.Google Scholar
  54. 54.
    Ingmar Baumgart, Bernhard Heep, Stephan Krause. OverSim: A Flexible Overlay Network Simulation Framework. Proceedings of 10th IEEE Global Internet Symposium (GI ‘07) in conjunction with IEEE INFOCOM 2007. 2007.Google Scholar
  55. 55.
    H. Casanova, A. Legrand, M. Quinson. Simgrid: a Generic Framework for Large-Scale Distributed Experiments. 10th IEEE International Conference on Computer Modeling and Simulation. 2008.Google Scholar
  56. 56.
    Chunqiang Tang. DSF: A Common Platform for Distributed Systems Research and Development. In Proc. of Middleware 2009, pp. 414–436. 2009.Google Scholar
  57. 57.
    Marcel Dischinger, Andreas Haeberlen, Ivan Beschastnikh, P. Krishna Gummadi, Stefan Saroiu. Satellitelab: Adding Heterogeneity to Planetary-scale Network Test-beds. SIGCOMM 2008: 315–326. 2008.Google Scholar
  58. 58.
    F. Brasileiro, E. Araujo, W. Voorsluys, M. Oliveira, F. Figueiredo. Bridging the High Performance Computing Gap: the Ourgrid Experience. CCGRID 2007 Seventh IEEE International Symposium on Cluster Computing and the Grid. Rio de Janeiro, Brazil. 2007.Google Scholar
  59. 59.
    Yuanyuan Zhou, Liviu Iftode, Jaswinder Pal Singh, Kai Li, Brian R. Toonen, Ioannis Schoinas, Mark D. Hill, David A. Wood. Relaxed Consistency and Coherence Granularity in DSM Systems: A Performance Evaluation. Sixth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 1997.Google Scholar
  60. 60.
    Maurice Herlihy, Victor Luchangco, Mark Moir, and William N. Scherer III. Software Transactional Memory for Dynamic-Sized Data Structures. Proceedings of the Twenty-Second Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing 2003. 2003.Google Scholar
  61. 61.
    Terry, D.B., Theimer, M.M., Petersen, K., Demers, A.J., Spreitzer, M.J., Hauser, C.. Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System. Proceedings of the fifteenth ACM Symposium on Operating Systems Principles. 1995.Google Scholar
  62. 62.
    H. Yu, A. Vahdat. The Costs and Limits of Availability for Replicated Services. ACM Transactions on Computer Systems. 2006.Google Scholar
  63. 63.
    J. Chen, B. Wu, M. Delap, B. Knutsson, H. Lu, C. Amza. Locality Aware Dynamic Load Management for Massively Multiplayer Games. Proceedings of the tenth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 2005.Google Scholar
  64. 64.
    V. Lo, D. Zappala, D. Zhou, Y. Liu, and S. Zhao. Cluster Computing on the Fly: P2P Scheduling of Idle Cycles in the Internet. In The 3rd International Workshop on Peer-to-Peer Systems (IPTPS’04) 2004.Google Scholar
  65. 65.
    S. Androutsellis-Theotokis, D. Spinellis A Survey of Peer-to-Peer Content Distribution Technologies. ACM Computing Surveys, vol. 36–4, p. 371. 2004.Google Scholar
  66. 66.
    M. Castro, M. Costa, A. Rowstron. Debunking some Myths about Structured and Unstructured Overlays. Symposium on Networked Systems Design & Implementation. Volume 2, p. 98. 2005.Google Scholar
  67. 67.
    P. Trunfio, D. Talia, H. Papadakis, P. Fragopoulou, M. Mordacchini, M. Pennanen, K. Popov, V. Vlassov, S. Haridi. Peer-to-Peer Resource Discovery in Grids: Models and Systems. Future Generation Computer Systems, archive Volume 23. 2007.Google Scholar
  68. 68.
    M. Cai, M. Frank, J. Chen, P. Szekely. MAAN: A Multi-attribute Addressable Network for Grid Information Services. Proc. 4th Int. Workshop on Grid Computing, GRID 2003. 2003.Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Luís Veiga
    • 1
  • João Nuno Silva
    • 1
  • João Coelho Garcia
    • 1
  1. 1.INESC ID Lisboa/Technical University of LisbonLisbonPortugal

Personalised recommendations