Advertisement

Dynamic Right-Sizing: An Automated, Lightweight, and Scalable Technique for Enhancing Grid Performance

  • Wu-chun Feng
  • Mike Fisk
  • Mark Gardner
  • Eric Weigle
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2334)

Abstract

With the advent of computational grids, networking performance over the wide-area network (WAN) has become a critical component in the grid infrastructure. Unfortunately, many high-performance grid applications only use a small fraction of their available bandwidth because operating systems and their associated protocol stacks are still tuned for yesterday’s WAN speeds. As a result, network gurus undertake the tedious process of manually tuning system buffers to allow TCP flow control to scale to today’s WAN grid environments. And although recent research has shown how to set the size of these system buffers automatically at connection set-up, the buffer sizes are only appropriate at the beginning of the connection’s lifetime. To address these problems, we describe an automated and lightweight technique called dynamic right-sizing that can improve throughput by as much as an order of magnitude while still abiding by TCP semantics.

Keywords

User Space Congestion Window Bottleneck Link File Transfer Protocol Bottleneck Bandwidth 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Foster, I. and Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Morgan-Kaufmann Publishers. San Francisco, California (1998).Google Scholar
  2. 2.
    Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International Journal of Supercomputer Applications (2001).Google Scholar
  3. 3.
    Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.: The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. International Journal of Supercomputer Applications (2001).Google Scholar
  4. 4.
    Argonne National Laboratory, California Institute of Technology, Lawrence Berkeley National Laboratory, Stanford Linear Accelerator Center, Jefferson Laboratory, University of Wisconsin, Brookhaven National Laboratory, Fermi National Laboratory, and San Diego Supercomputing Center: The Particle Physics Data Grid. http://www.cacr.caltech.edu/ppdg/.
  5. 5.
    Childers, L., Disz, T., Olson, R. Papka, M., Stevens, R., Udeshi, T.: Access Grid: Immersive Group-to-Group Collaborative Visualization. Proceedings of the 4th International Immersive Projection Workshop (2000).Google Scholar
  6. 6.
    Partridge, C., Shepard, T.: TCP/IP Performance over Satellite Links. IEEE Network. 11 (1997) 44–49.CrossRefGoogle Scholar
  7. 7.
    Allman, M., Glover, D., Sanchez, L.: Enhancing TCP Over Satellite Channels Using Standard Mechanisms. IETF RFC 2488 (1999).Google Scholar
  8. 8.
    Allman, M. et al.: Ongoing TCP Research Related to Satellites. IETF RFC 2760 (2000).Google Scholar
  9. 9.
    Feng, W., Tinnakornsrisuphap, P.: The Failure of TCP in High-Performance Computational Grids. Proceedings of SC 2000: High-Performance Networking and Computing Conference (2000).Google Scholar
  10. 10.
    Pittsburgh Supercomputing Center. Enabling High-Performance Data Transfers on Hosts. http://www.psc.edu/networking/perf_tune.html.
  11. 11.
    Tierney, B. TCP Tuning Guide for Distributed Applications on Wide-Area Networks. USENIX & SAGE Login. http://www-didc.lbl.gov/tcp-wan.html (2001).
  12. 12.
    Tirumala, A. and Ferguson, J.: IPERF Version 1.2. http://dast.nlanr.net/Projects/Iperf/index.html (2001).
  13. 13.
    Lai, K., Baker, M.: Nettimer: A Tool for Measuring Bottleneck Link Bandwidth. Proceedings of the USENIX Symposium on Internet Technologies and Systems (2001).Google Scholar
  14. 14.
    University of Kansas, Information & Telecommunication Technology Center: NetSpec: A Tool for Network Experimentation and Measurement. http://www.ittc.ukans.edu/netspec.
  15. 15.
    Lawrence Berkeley National Laboratory: Nettest: Secure Network Testing and Monitoring. http://www-itg.lbl.gov/nettest.
  16. 16.
    Mah, B.: pchar: A Tool for Measuring Internet Path Characteristics. http://www.employees.org/bmah/Software/pchar.
  17. 17.
    Jin., G., Yang, G., Crowley, B., Agrawal, D.: Network Characterization Service. Proceedings of the IEEE Symposium on High-Performance Distributed Computing (2001).Google Scholar
  18. 18.
    Liu, J., Ferguson, J.: Automatic TCP Socket Buffer Tuning. Proceedings of SC 2000: High-Performance Networking and Computing Conference (2000).Google Scholar
  19. 19.
    Tierney, B., Gunter, D., Lee, J., Stoufer, M.: Enabling Network-Aware Applications. Proceedings of the IEEE International Symposium on High-Performance Distributed Computing (2001).Google Scholar
  20. 20.
    National Center for Atmospheric Research, Pittsburgh Supercomputing Center, and National Center for Supercomputing Applications. The Web100 Project. http://www.web100.org.
  21. 21.
  22. 22.
    Semke, J., Mahdavi, J., Mathis, M. Automatic TCP Buffer Tuning. Proceedings of ACM SIGCOMM (1998).Google Scholar
  23. 23.
    Fisk, M., Feng, W.: Dynamic Adjustment of TCP Window Sizes. Los Alamos National Laboratory Unclassified Report, LA-UR 00-3221 (2000).Google Scholar
  24. 24.
    Fisk, M., Feng, W.: Dynamic Right-Sizing: TCP Flow-Control Adaptation (poster). Proceedings of SC 2001: High-Performance Networking and Computing Conference (2001).Google Scholar
  25. 25.
    Dunigan, T., Fowler, F. Personal Communication with Web100 Project (2002).Google Scholar
  26. 26.
    Jacobson, V., Braden, R., Borman, D.: TCP Extensions for High Performance. IETF RFC 1323 (1992).Google Scholar
  27. 27.
    Postel, J., Reynolds, J. File Transfer Protocol (FTP). IETF RFC 959 (1985).Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Wu-chun Feng
    • 1
  • Mike Fisk
    • 1
  • Mark Gardner
    • 1
  • Eric Weigle
    • 1
  1. 1.Los Alamos National LaboratoryResearch & Development in Advanced Network Technology (RADIANT)Los AlamosUSA

Personalised recommendations