Skip to main content
SpringerLink
Log in

Managing Long Running Queries in Grid Environment

  • Conference paper
On the Move to Meaningful Internet Systems 2004: OTM 2004 Workshops (OTM 2004)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3292))

Abstract

Exceptionally large amounts of both distributed data and computational resources are becoming available through the Grid. This will enable efficient exchange and processing of very large amounts of data combined with CPU intensive computations, as required by many scientific applications. We propose a customizable Grid-based query processor built on top of an established Grid infrastructure, NorduGrid. It allows users to submit queries wrapping user-defined long running operations that filter and transform distributed customized data. Limitations imposed by the used Grid infrastructure influence the architecture. For example, resource requirements have to be specified before Grid jobs are started and delays may occur based on the availability of required resources for a job. We are developing a fully functional prototype to investigate the viability of the approach and its applicability. Our first application area is Particle Physics where scientists analyze huge amount of data produced by a collider or simulators to identify particles.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. ATLAS Data Challenges (2001), [Online], http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/DC/doc/AtlasDCs.pdf

  2. ATLAS Data Challenge 1, Draft 3.0 (2003), [Online], http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/DC/DC1/DC1-Report -V3[1].0-22092003.pdf

  3. Bethke, S., Calvetti, M., Hoffmann, H.F., Jacobs, D., Kasemann, M., Linglin, D.: Report of the Steering Group of the LHC Computing Review, CERN/LHCC/2001–004 (2001), Available at http://lhcb-comp.web.cern.ch/lhcb-comp/Reviews/LHCComputing2000/Report_final.pdf

  4. Bukhres, O., Elmagarmid, A. (eds.): Object-Oriented Multidatabase Systems: a Solution for Advanced Applications. Prentice Hall, Englewood Cliffs (1996)

    Google Scholar 

  5. Bouganim, L., Fabret, F., Porto, F., Valduriez, P.: Processing Queries with Expensive Functions and Large Objects in Distributed Mediator Systems. In: Proc. of Int. Conf. on Data Engineering, ICDE (2001)

    Google Scholar 

  6. Bisset, M., Moortgat, F., Moretti, S.: Trilepton+top signal from chargino-neutralino decays of MSSM charged Higgs bosons at the LHC. Eur.Phys.J. C30, 419–434 (2003)

    Google Scholar 

  7. Brun, R., Rademakers, F.: ROOT – An Object Oriented Data Analysis Framework. In: Proceedings AIHENP 1996 Workshop, Lausanne (September 1996); Nucl. Inst. & Meth. in Phys. Res. A 389, 81–86 (1997)

    Google Scholar 

  8. Condor-G System, [Online], http://www.cs.wisc.edu/condor/condorg/

  9. Czajkowski, K., Foster, I., Karonis, N., Kesselman, C., Martin, S., Smith, W., Tuecke, S.: A Resource Management Architecture for Metacomputing Systems. In: Proc. IPPS/SPDP 1998 Workshop on Job Scheduling Strategies for Parallel Processing, pp. 62–82 (1998)

    Google Scholar 

  10. Directed Acyclic Graph Manager, [Online], http://www.cs.wisc.edu/condor/dagman/

  11. Eerola, P., Ekelof, T., Ellert, M., Hansen, J.R., Hellman, S., Konstantinov, A., Kónya, B., Myklebust, T., Nielsen, J.L., Ould-Saada, F., Smirnova, O., Wäänänen, A.: Atlas Data- Challenge 1 on NorduGrid, ECONF C0303241:MOCT011 (2003)

    Google Scholar 

  12. Elmasri, R., Navathe, S.B.: Enhanced Entity-Relationship and UML Modeling. In: Fundamentals of Database Systems, 4th edn., pp. 85–121. Addison-Wesley, Reading (2004)

    Google Scholar 

  13. Flodin, S., Hansson, M., Josifovski, V., Katchaounov, T., Risch, T., Sköld, M.: Amos II Release 6 User’s Manual, [Online] (2004), http://user.it.uu.se/~udbl/amos/doc/amos_users_guide.html

  14. Foster, I., Kesselman, C.: Globus: A Toolkit-Based Grid Architecture. In: The Grid: Blueprint for a New Computing Infrastructure, pp. 259–278. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  15. Fahl, G., Risch, T.: Query Processing over Object Views of Relational Data. The VLDB Journal 6(4), 261–281 (1997), Available at http://www.dis.uu.se/~udbl/publ/vldbj97.pdf

    Article  Google Scholar 

  16. Frey, J., Tannenbaum, T., Foster, I., Livny, M., Tuecke, S.: Condor-G: A Computation Management Agent for Multi-Institutional Grids. In: Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC), San Francisco, California, pp. 7–9 (2001)

    Google Scholar 

  17. Globus I/O API, [Online], http://www-unix.globus.org/api/c-globus-3.2/globus_io/html/

  18. Gounaris, A., Paton, N.W., Fernances, A.A.A., Sakellariou, R.: Adaptive Query Processing: A Survey. In: Proceedings of the 19th British National Conference on Databases: Advances in Databases (2002)

    Google Scholar 

  19. Garcia-Molina, H., Papakonstantinou, Y., Quass, D., Rajaraman, A., Sagiv, Y., Ullman, J., Vassalos, V., Widom, J.: The TSIMMIS Approach to Mediation: Data Models and Languages. Journal of Intelligent Information Systems (JIIS) 8(2), 117–132 (1997)

    Article  Google Scholar 

  20. Garcia-Molina, H., Salem, K.: Main Memory Database Systems: An Overview. IEEE Transactions on Knowledge and Data Engineering 4(6), 509–516 (1992)

    Article  Google Scholar 

  21. Haas, L., Kossmann, D., Wimmers, E.L., Yang, J.: Optimizing Queries across Diverse Data Sources. In: 23rd Intl. Conf. on Very Large Databases (VLDB 1997), pp. 276–285 (1997)

    Google Scholar 

  22. Josifovski, V., Risch, T.: Query Decomposition for a Distributed Object-Oriented Mediator System. Distributed and Parallel Databases, J. Kluwer, 307–336 (2002), Available at http://user.it.uu.se/~torer/publ/dpdb.pdf

  23. Karlsson, J.S., Lal, A., Leung, C., Pham, T.: IBM DB2 Everyplace: A Small Footprint Relational Database System. In: 17th International Conference on Data Engineering (2001)

    Google Scholar 

  24. Konstantinov, A.: The NorduGrid Grid Manager and GridFTP Server, Description and Administrator’s Manual, [Online] (2004), http://www.nordugrid.org/papers.html

  25. Kónya, B.: The NorduGrid Information System, [Online] (2002), http://www.nordugrid.org/documents/ng-infosys.pdf

  26. The Large Hadron Collider, [Online], http://lhc-new-homepage.web.cern.ch/lhc-new-homepage/

  27. Liu, L., Pu, C.: An Adaptive Object-Oriented Approach to Integration and Access of Heterogeneous Information Sources. Distributed and Parallel Databases 5(2), 167–205 (1997)

    Article  Google Scholar 

  28. Lin, H., Risch, T., Katchanounov, T.: Adaptive data mediation over XML data. In: special issue on Web Information Systems Applications. Journal of Applied System Studies (JASS), Cambridge International Science Publishing, 3(2) (2002), Available at http://user.it.uu.se/~torer/publ/jass01.pdf

  29. The Message Passing Interface (MPI) Standard, [Online], http://www-unix.mcs.anl.gov/mpi/

  30. The NorduGrid User Guide, [Online] (2004), http://www.nordugrid.org/documents/userguide.pdf

  31. NorduGrid middleware, the Advance Resource Connector, [Online], http://www.nordugrid.org/middleware/

  32. ORACLE Inc.: Oracle8i Lite: The Internet Platform for Mobile Computing

    Google Scholar 

  33. Özsu, M.T., Valduriez, P.: Principles of Distributed Database Systems, 2nd edn. Prentice Hall, Englewood Cliffs (1999)

    Google Scholar 

  34. Quass, D., Rajaraman, A., Sagiv, Y., Ullman, J., Widom, J.: Querying Semistructured Heterogeneous Information. In: Ling, T.-W., Vieille, L., Mendelzon, A.O. (eds.) DOOD 1995. LNCS, vol. 1013, pp. 319–344. Springer, Heidelberg (1995)

    Google Scholar 

  35. Risch, T., Josifovski, V.: Distributed Data Integration by Object-Oriented Mediator Servers. Concurrency and Computation: Practice and Experience J. 13(11), John Wiley & Sons (September 2001), Available at http://user.it.uu.se/~udbl/publ/ddiooms.pdf

  36. ROOT: An Object-Oriented Data Analysis Framework – User’s Guide, [Online] (2004), http://root.cern.ch/root/doc/RootDoc.html

  37. Swegrid, [Online], www.swegrid.se

  38. SYBASE Inc.: SQL Anywhere Studio, http://www.sybase.com/mobile/

  39. Smirnova, O.: Extended Resource Specification Language, [Online] (2003), http://www.nordugrid.org/documents/xrsl.pdf

  40. Stonebraker, M., Brown, P.: Object-Relational DBMSs: Tracking the Next Great Wave, 2nd edn. Morgan Kaufmann Publishers, San Francisco (1999)

    Google Scholar 

  41. Smirnova, O., Eerola, P., Ekelöf, T., Ellert, M., Hansen, J.R., Konstantinov, A., Kónya, B., Nielsen, J.L., Ould-Saada, F., Wäänänen, A.: The NorduGrid Architecture and Middleware for Scientific Applications. In: International Conference on Computational Science, pp. 264–273 (2003)

    Google Scholar 

  42. Smith, J., Gounaris, A., Watson, P., Paton, N.W., Fernandes, A.A.A., Sakellariou, R.: Distributed Query Processing on the Grid. In: Parashar, M. (ed.) GRID 2002. LNCS, vol. 2536, pp. 279–290. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  43. Tomasic, L., Raschid, P.: Valduriez: Scaling Access to Heterogeneous Data Sources with DISCO. IEEE Transactions on Knowledge and Date Engineering 10(5), 808–823 (1998)

    Article  Google Scholar 

  44. Wiederhold, G.: Mediators in the architecture of future information systems. IEEE Computer 25(3), 38–49 (1992)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Uppsala Database Laboratory, Department of Information Technology, Uppsala University, P.O. Box 337, SE-751 05, Uppsala, Sweden

    Ruslan Fomkin & Tore Risch

Authors
  1. Ruslan Fomkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tore Risch
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Vrije Universiteit Brussel (VUB), STARLab, Bldg G/10, Pleinlaan 2, 1050, Brussels, Belgium

    Robert Meersman

  2. School of Computer Science and Information Technology, RMIT University, Bld 10.10, 376-392 Swanston Street, 3001, Melbourne, VIC, Australia

    Zahir Tari

  3. PrismTech, 4, Rue Angiboust, 91460, Marcoussis, France

    Angelo Corsaro

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fomkin, R., Risch, T. (2004). Managing Long Running Queries in Grid Environment. In: Meersman, R., Tari, Z., Corsaro, A. (eds) On the Move to Meaningful Internet Systems 2004: OTM 2004 Workshops. OTM 2004. Lecture Notes in Computer Science, vol 3292. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30470-8_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-30470-8_28

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-23664-1

  • Online ISBN: 978-3-540-30470-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation