Data Management in an International Data Grid Project

  • Wolfgang Hoschek
  • Javier Jaen-Martinez
  • Asad Samar
  • Heinz Stockinger
  • Kurt Stockinger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1971)


In this paper we report on preliminary work and architectural design carried out in the “Data Management” work package in the International Data Grid project. Our aim within a time scale of three years is to provide Grid middleware services supporting the I/O-intensive world-wide distributed next generation experiments in High-Energy Physics, Earth Observation and Bioinformatics. The goal is to specify, develop, integrate and test tools and middleware infrastructure to coherently manage and share Petabyte-range information volumes in high-throughput production-quality Grid environments. The middleware will allow secure access to massive amounts of data in a universal name-space, to move and replicate data at high speed from one geographical site to another, and to manage synchronisation of remote copies. We put much attention on clearly specifying and categorising existing work on the Grid, especially in data management in Grid related projects. Challenging use cases are described and how they map to architectural decisions concerning data access, replication, meta data management, security and query optimisation.


Large Hadron Collider Storage System Data Grid Access Pattern Meta Data 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    T. Anderson, Y. Breitbart, H. Korth, A. Wool. Replication, Consistency, and Practicality: Are These Mutually Exclusive? Proc. SIGMOD International Conference on the Management of Data, pp. 484–495 1998.Google Scholar
  2. 2.
    O. Barring, J. Baud, J. Durand. CASTOR Project Status, Proc. of Computing in High Energy Physics 2000, Padova, Febr. 2000.Google Scholar
  3. 3.
    J. Bester, I. Foster, C. Kesselman, J. Tedesco, S. Tuecke. GASS: A Data Movement and Access Service for Wide Area Computing Systems. In Proceedings of the Sixth Workshop on I/O in Parallel and Distributed Systems, May 1999.Google Scholar
  4. 4.
    A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, S. Tuecke. The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific DataSets. Network Storage Symposium, Seattle 1999.Google Scholar
  5. 5.
    P. Corbett and D. Feitelson. Design and Implementation of the Vesta Parallel File System. In Proceedings of the Scalable High-Performance Computing Conference, pages 63–70, 1994.Google Scholar
  6. 9.
    K. Holtman, H. Stockinger. Building a Large Location Table to Find Replicas of Physics Objects. Proc. of Computing in High Energy Physics 2000, Padova, Febr. 2000.Google Scholar
  7. 11.
    W. Johnston, J. Lee, B. Tierney, C. Tull, D. Millsom. The China Clipper Project: A Data Intensive Grid Support for Dynamically Configured, Adaptive, Distributed, High-Performance Data and Computing Environments. Proc. of Computing in High Energy Physics 1998, Chicago 1998.Google Scholar
  8. 12.
    W. Johnston, D. Gannon, B. Nitzberg. Grids as Production Computing Environments: The Engineering Aspects of NASA’s Information Power Grid. Eighth IEEE International Symposium on High Performance Distributed Computing, Redondo 1999.Google Scholar
  9. 15.
    J. Morris, et al. Andrew: A Distributed Personal Computing Evironment. Comms. ACM, vol 29, no. 3, pp. 184–201, 1996.CrossRefGoogle Scholar
  10. 16.
    N. Nieuwejaar, D. Kotz. The Galley Parallele File System. In Proceedings of the 10th ACM International Conference on Supercomputing, pages 374–381, Philadelphia, ACM Press, May 1996.Google Scholar
  11. 18.
    R. Sandberg. The Sun Network File System: Design, Implementation and Experience, Tech. Report, Mountain View CA: Sun Microsystems, 1987.Google Scholar
  12. 19.
    H. Stockinger, Data Replication in Distributed Database Systems, CMS Note 1999/046, Geneva, July 1999.Google Scholar
  13. 20.
    K. Stockinger, D. Duellmann, W. Hoschek, E. Schikuta. Improving the Performance of High Energy Physics Analysis through Bitmap Indices. To appear in DEXA’200, Springer Verlag, Sept. 2000.Google Scholar
  14. 22.
    W. Yeong, T. Howes, S. Kille. Lightweight Directory Access Protocol, RFC 1777. Performance Systems International, University of Michigan, ISODE Consortium, March 1995.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Wolfgang Hoschek
    • 1
    • 3
  • Javier Jaen-Martinez
    • 1
  • Asad Samar
    • 1
    • 4
  • Heinz Stockinger
    • 1
    • 2
  • Kurt Stockinger
    • 1
    • 2
  1. 1.CERNEuropean Organization for Nuclear ResearchGenevaSwitzerland
  2. 2.Inst. for Computer Science and Business InformaticsUniversity of ViennaAustria
  3. 3.Inst. of Applied Computer ScienceUniversity of LinzAustria
  4. 4.California Institute of TechnologyCAUSA

Personalised recommendations