Skip to main content
SpringerLink
Log in

Storage and Analysis Infrastructure for Data Acquisition Systems with High Data Rates

  • Chapter
  • First Online:
Remote Instrumentation Services on the e-Infrastructure

  • 313 Accesses

Abstract

The purpose of process data processing is the analysis of process data. These data are normally measured using a data acquisition system. Several projects use distributed systems, making the controlling from a central instance with secure access to the infrastructure necessary. Special for several projects are also very high resource demands, e.g. the produced data rates. In order to fulfill the requirements of such projects the development of a data acquisition system is necessary. The main requirements are the secure access to the infrastructure, handling of high data rates and the ad hoc analysis of the measured data. In order to make the system scalable and reliable it should be based on open standards. This chapter evaluates already existing remote control software packages and data management systems for the development of distributed data acquisition systems to fulfill the requirements of process data processing projects. The evaluation identifies that the usage of Grid technologies provides the necessary features as the Grid offers, e.g., Grid security and access to storage and computation infrastructure. The provided architecture uses the GRIDCC framework for controlling the detectors, the communication with data management systems to handle the amount of measured data and the communication with Service Oriented Architectures for ad hoc analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. L. Abadie, P. Badino, J.-P. Baud, J. Casey, A. Frohner, G. Grosdidier, S. Lemaitre, G. Mccance, R. Mollon, K. Nienartowicz, D. Smith, and P. Tedesco. Grid–Enabled Standards–based Data Management. 24th IEEE Conference on Mass Storage Systems and Technologies, 2007. MSST 2007., pp. 60–71, September 2007

    Google Scholar 

  2. J. Abraham, M. Aglietta, I.C. Aguirre, et al. Properties and performance of the prototype instrument for the pierre auger observatory. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 523(1-2),50–95, 2004

    Article  Google Scholar 

  3. W. Allcock, J. Bresnahan, R. Kettimuthu, and M. Link. The Globus Striped GridFTP Framework and Server. Supercomputing, 2005. Proceedings of the ACM/IEEE SC 2005 Conference, pp. 54–54, November 2005

    Google Scholar 

  4. C. Baru, R. Moore, A. Rajasekar, and M. Wan. The SDSC storage resource broker. CASCON ’98: Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research, 5, IBM Press, 1998

    Google Scholar 

  5. J. D. Bregman. Towards a concept design for a LOFAR. The Universe at Low Radio Frequencies, Proceedings of IAU Symposium 199, 484, 2002

    Google Scholar 

  6. J.-M. Chaize, A. Götz, W.-D. Klotz, J. Meyer, M. Perez, and E. Taurel. TANGO an object oriented control system based on CORBA. ICALEPCS, 1999

    Google Scholar 

  7. P. Daponte, D. Grimaldi, and M. Marinov. Real–time measurement and control of an industrial system over a standard network: Implementation of a prototype for educational purposes. IEEE Transactions on Instrumentation and Measurement, 51(5),962–969, October 2002

    Article  Google Scholar 

  8. dCache.org. http://www.dcache.org/manuals/Book September 2010. The dCache Book

  9. A. Depari, P. Ferrari, A. Flammini, D. Marioli, and A. Taroni. A new instrument for real-time ethernet performance measurement. IEEE Transactions on Instrumentation and Measurement, 57(1),121–127, January 2008

    Article  Google Scholar 

  10. F. Donno, A. Ghiselli, L. Magnoni, and R. Zappi. StoRM: GRID middleware for disk resource management. Proceedings of the International CHEP 2004, Interlaken, Switzerland, October 2004

    Google Scholar 

  11. M. Ernst, P. Fuhrmann, M. Gasthuber, T. Mkrtchyan, and C. Waldman. dCache, a Distributed Storage Data Caching System. Proceedings of the International CHEP 2001, Beijing, September 2001

    Google Scholar 

  12. I. Foster. Globus toolkit version 4: Software for service-oriented systems. IFIP International Conference on Network and Parallel Computing, LNCS 3779, Springer, pp. 2–13, 2005

    Google Scholar 

  13. G. Giorgi and C. Narduzzi. Detection of anomalous behaviors in networks from traffic measurements. IEEE Transactions on Instrumentation and Measurement, 57(12),2782–2791, December 2008

    Article  Google Scholar 

  14. A. Götz, W.-D. Klotz, P. Mäkijärvi, J. Meyer, E. Taurel, and J. Quick. TACO: An object oriented system for PC’s running Linux, Windows/NT, OS-9, LynxOS or VxWorks. The second workshop on PCs and Particle Accelerator Control, Tsukuba, 1999

    Google Scholar 

  15. M. A. Howe, G. A. Cox, P. J. Harvey, F. McGirt, K. Rielage, J. F. Wilkerson, and J. M. Wouters. Sudbury neutrino observatory neutral current detector acquisition software overview. IEEE Transactions on Nuclear Science, 51(3),878–883, June 2004

    Article  Google Scholar 

  16. ITER Technical Basis. ITER EDA Documentation Series, 24, IAEA Vienna 2002

    Google Scholar 

  17. KATRIN Collaboration. KATRIN Design Report 2004. Technical Report FZKA-7090, February 2005

    Google Scholar 

  18. R. Lange, A. Johnson, M.R. Kraimer, W.E. Norum, and J. Hill. Epics: Recent developments and future perspectives. 9th International Conference on Accelerator and Large Experimental Physics Control Systems, 278, 2003

    Google Scholar 

  19. Lawrence Berkeley National Laboratory Scientific Data Management Research Group. BeStMan, http://datagrid.lbl.gov/bestman/ September 2010

  20. F. Lelli, E. Frizziero, M. Gulmini, G. Maron, S. Orlando, A. Petrucci, and S. Squizzato. The many faces of the integration of instruments and the grid. International Journal of Web and Grid Services 2007, 3(3),239–266, 2007

    Article  Google Scholar 

  21. G. Lo Presti, O. Barring, A. Earl, R. Garcia Rioja, S. Ponce, G. Taurelli, D. Waldron, and M. Coelho Dos Santos. CASTOR: A distributed storage resource facility for high performance data processing at CERN. 24th IEEE Conference on Mass Storage Systems and Technologies, 2007. MSST 2007., pp. 275–280, September 2007

    Google Scholar 

  22. National Instruments. NI LabVIEW - The Software That Powers Virtual Instrumentation - Products and Services - National Instrument, http://www.ni.com/labview/ September 2010

  23. T. O’Reilly, D. Edgington, D. Davis, R. Henthorn, M. McCann, T. Meese, W. Radochonski, M. Risi, B. Roman, and R. Schramm. “Smart network” infrastructure for the MBARI ocean observing system. OCEANS, 2001. MTS/IEEE Conference and Exhibition, 2,1276–1282, 2001

    Google Scholar 

  24. R. Pugliese, F. Asnicar, L.D. Cano, L. Chittaro, R. Ranon, L.D. Marco, and A. Senerchia. Collaborative Environments For The GRID: The GRIDCC Multipurpose Collaborative Environment, chapter IV. Springer, New York, NY, 2006

    Google Scholar 

  25. A. Rajasekar, M. Wan, R. Moore, and W. Schroeder. A prototype rule-based distributed data management system. HPDC workshop on “Next Generation Distributed Data Management”, May 2006

    Google Scholar 

  26. RTEMS Steering Committee. Welcome to the RTEMS home page, http://www.rtems.com/ September 2010

  27. N. Ruiter, M. Zapf, R. Stotzka, et al. First Images with a 3D–Prototype for Ultrasound Computer Tomography. IEEE International Ultrasonics Symposium, 2005

    Google Scholar 

  28. A. Shoshani, A. Sim, and J. Gu. Storage resource managers. Grid Resource Management: State of the Art and Future Trends 1st edition, chapter 20, pp. 321–340. Springer, 2003

  29. O. Tatebe, Y. Morita, S. Matsuoka, N. Soda, and S. Sekiguchi. Grid datafarm architecture for petascale data intensive computing. 2nd IEEE/ACM International Symposium on, Cluster Computing and the Grid, 2002. May 2002

    Google Scholar 

  30. The EGEE project. EGEE: Enabling Grids for E-sciencE; http://www.eu-egee.org/ September 2010

  31. The xrootd team. The Scalla Software Suite: xrootd/cmsd, http://xrootd.slac.stanford.edu/ September 2010

  32. Wikipedia – The Free Encyclopedia. Welcome to Wikipedia, http://en.wikipedia.org September 2010

  33. Wind River Systems. Wind River; http://www.windriver.com/ September 2010

Download references

Author information

Authors and Affiliations

  1. Karlsruhe Institute of Technology, Institute for Data Processing and Electronics, Hermann-von-Helmholtz-Platz 1, 76344, Eggenstein-Leopoldshafen, Germany

    M. Sutter

  2. Karlsruhe Institute of Technology, Institute for Data Processing and Electronics, Eggenstein-Leopoldshafen, Germany

    T. Jejkal, R. Stotzka & V. Hartmann

  3. Karlsruhe Institute of Technology, Steinbuch Centre for Computing, Eggenstein-Leopoldshafen, Germany

    M. Hardt

Authors
  1. M. Sutter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. Jejkal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Stotzka
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. V. Hartmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. M. Hardt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Sutter .

Editor information

Editors and Affiliations

  1. DIST, University of Genoa, Via Opera Pia 13, Genova, 16145, Italy

    Franco Davoli

  2. Networking Center, Poznan Supercomputing &, ul. Noskowskiego 10, Poznan, 61-704, Poland

    Norbert Meyer

  3. Sincrotrone Trieste S.C.p.A, Strada Statale 14, Basovizza, Trieste, 34012, Italy

    Roberto Pugliese

  4. DIST, University of Genoa, Via Opera Pia 13, Genova, 16145, Italy

    Sandro Zappatore

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Sutter, M., Jejkal, T., Stotzka, R., Hartmann, V., Hardt, M. (2011). Storage and Analysis Infrastructure for Data Acquisition Systems with High Data Rates. In: Davoli, F., Meyer, N., Pugliese, R., Zappatore, S. (eds) Remote Instrumentation Services on the e-Infrastructure. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-5574-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-5574-6_7

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4419-5573-9

  • Online ISBN: 978-1-4419-5574-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation