Advertisement

Redundant Parallel File Transfer with Anticipative Recursively-Adjusting Scheme in Data Grids

  • Chao-Tung Yang
  • Yao-Chun Chi
  • Tsu-Fen Han
  • Ching-Hsien Hsu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4494)

Abstract

The co-allocation architecture was developed to enable the parallel download of datasets/servers from selected replica servers, and the bandwidth performance is the main factor that affects the internet transfer between the client and the server. Therefore, it is important to reduce the difference of finished time among replica servers, and manage changeful network performance during the term of transferring as well. In this paper, we proposed an Anticipative Recursively-Adjusting Co-Allocation scheme, to adjust the workload of each selected replica server, which handles unwarned variant network performances of the selected replica servers. The algorithm is based on the previous finished rate of assigned transfer size, to anticipate that bandwidth status on next section for adjusting the workload, and further, to reduce file transfer time in a grid environment. Our approach is usefully in unstable gird environment, which reduces the wasted idle time for waiting the slowest server and decreases file transfer completion time.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Allcock, B., Bester, J., Bresnahan, J., Chervenak, A., Foster, I., Kesselman, C., Meder, S., Nefedova, V., Quesnel, D., Tuecke, S.: Data Management and Transfer in High-Performance Computational Grid Environments. Parallel Computing 28, 749–771 (2002)CrossRefGoogle Scholar
  2. 2.
    Bhuvaneswaran, R.S., Katayama, Y., Takahashi, N.: Dynamic Co-allocation Scheme for Parallel Data Transfer in Grid Environment. In: Proceedings of First International Conference on Semantics, Knowledge, and Grid (SKG 2005), vol. 17, IEEE CS Press, Los Alamitos (2005)Google Scholar
  3. 3.
    Chen, C.H., Yang, C.T., Lai, C.L.: Towards an Efficient Replica Selection for Data Grid, In: Proceedings of the First Workshop on Grid Technologies and Applications (WoGTA 2004) pp. 89–94 (2004)Google Scholar
  4. 4.
    Chervenak, A., Deelman, E., Foster, I., Guy, L., Hoschek, W., Iamnitchi, A., Kesselman, C., Kunszt, P., Ripeanu, M.: Giggle: A Framework for Constructing Scalable Replica Location Services. In: Proceedings of the 2002 ACM/IEEE conference on Supercomputing, pp. 1–17 (2002)Google Scholar
  5. 5.
    Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.: The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. Journal of Network and Computer Applications 23, 187–200 (2001)CrossRefGoogle Scholar
  6. 6.
    Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid Information Services for Distributed Resource Sharing. In: Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing (HPDC-10’ 2001), pp. 181–194 (2001)Google Scholar
  7. 7.
    Czajkowski, K., Foster, I., Kesselman, C.: Resource Co-Allocation in Computational Grids. In: Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8’ 1999) (1999) Google Scholar
  8. 8.
    Hoschek, W., Jaen-Martinez, J., Samar, A., Stockinger, H., Stockinger, K.: Data Management in an International Data Grid Project. In: Proceedings of the First IEEE/ACM International Workshop on Grid Computing (2000)Google Scholar
  9. 9.
    Open Grid Forum, http://www.ogf.org
  10. 10.
    Stockinger, H., Samar, H., Allcock, A., Foster, B., Holtman, I., B., T.: File and Object Replication in Data Grids. Journal of Cluster Computing 5, 305–314 (2002)CrossRefGoogle Scholar
  11. 11.
    The Globus Alliance, http://www.globus.org
  12. 12.
    Vazhkudai, S.: Enabling the Co-Allocation of Grid Data Transfers. In: Proceedings of Fourth International Workshop on Grid Computing, pp. 44–51(2003)Google Scholar
  13. 13.
    Vazhkudai, S., Tuecke, S., Foster, I.: Replica Selection in the Globus Data Grid. In: Proceedings of the 1st International Symposium on Cluster Computing and the Grid (CCGRID 2001) pp. 106–113 (2001)Google Scholar
  14. 14.
    Vazhkudai, S., Schopf, J.M.: Using Regression Techniques to Predict Large Data Transfers. International Journal of High Performance Computing Applications 17, 249–268 (2003)CrossRefGoogle Scholar
  15. 15.
    Vazhkudai S., Schopf, J.M.: Predicting Sporadic Grid Data Transfers. In: Proceedings of 11th IEEE International Symposium on High Performance Distributed Computing (HPDC-11 2002) pp. 188–196 (2002)Google Scholar
  16. 16.
    Vazhkudai, S., Schopf, J.M., Foster I.: Predicting the Performance of Wide Area Data Transfers. In: Proceedings of the 16th International Parallel and Distributed Processing Symposium (IPDPS 2002) pp. 34–43 (2002)Google Scholar
  17. 17.
    Wang, C.M., Hsu, C.C., Chen, H.M., Wu, J.J.: Efficient Multi-Source Data Transfer in Data Grids. In: Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID 2006), pp. 421–424 (2006)Google Scholar
  18. 18.
    Yang, C.T., Yang, I.H., Chen, C.H.: Improve Dynamic Adjustment Mechanism in Co-Allocation Data Grid Environments. In: Proceedings of the 11th Workshop on Compiler Techniques for High-Performance Computing (CTHPC-11 2005) pp. 189–194 (2005)Google Scholar
  19. 19.
    Yang, C.T., Chen, C.H., Li, K.C., Hsu, C.H.: Performance Analysis of Applying Replica Selection Technology for Data Grid Environments. In: Hurd, J., Melham, T. (eds.) TPHOLs 2005. LNCS, vol. 3603, pp. 278–287. Springer, Berlin Heidelberg New York (2005)Google Scholar
  20. 20.
    Yang, C.T., Yang, I.H., Li, K.C., Hsu, C.H.: A Recursive-Adjustment Co-Allocation Scheme in Data Grid Environments. LNCS, vol. 3719, pp. 40–49. Springer-Verlag, Heidelberg (2005)Google Scholar
  21. 21.
    Yang, C.T., Yang, I.H., Li, K.C., Wang, S.Y.: Improvements on Dynamic Adjustment Mechanism in Co-Allocation Data Grid Environments. The Journal of Supercomputing (2007)Google Scholar
  22. 22.
    Yang, C.T., Yang, I.H., Chen, C.H., Wang, S.Y.: Implementation of a Dynamic Adjustment Mechanism with Efficient Replica Selection in Co-Allocation Data Grid Environments. In: Proceedings of the 21st Annual ACM Symposium on Applied Computing (SAC 2006) - Distributed Systems and Grid Computing (DSGC), Track, 1, pp. 797–804 (2006)Google Scholar
  23. 23.
    Yang, C.T., Wang, S.Y., Lin, C.H., Lee, M.H., Wu, T.Y.: Cyber-Transformer: A Toolkit for Files Transfer with Replica Management in Data Grid Environments. In: Proceedings of the Second Workshop on Grid Technologies and Applications (WoGTA 2005) pp. 73–80 (2005)Google Scholar
  24. 24.
    Yang, C.T., Wang, S.Y., Fu, C.P.: A Dynamic Adjustment Mechanism for Data Transfer in Data Grids. In: Proceeding of the third IFIP International Conference on Network and Parallel Computing (NPC 2006) pp. 110–120 (2006)Google Scholar
  25. 25.
    Yang, L., Schopf, J.M., Foster, I.: Improving Parallel Data Transfer Times Using Predicted Variances in Shared Networks. In: Proceedings of the fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2005) vol. 2, pp. 734–742 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Chao-Tung Yang
    • 1
  • Yao-Chun Chi
    • 1
  • Tsu-Fen Han
    • 1
  • Ching-Hsien Hsu
    • 2
  1. 1.High-Performance Computing Laboratory, Department of Computer Science and Information Engineering, Tunghai University, Taichung, 40704, TaiwanR.O.C
  2. 2.Department of Computer Science and Information Engineering, Chung Hua University, Hsinchu 300Taiwan

Personalised recommendations