Skip to main content
SpringerLink
Log in

Ontology-Based Integrated Monitoring of Hadoop Clusters in Industrial Environments with OPC UA and RESTful Web Services

  • Conference paper
  • First Online:
Computer Networks (CN 2015)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 522))

Included in the following conference series:

Abstract

Contemporary industrial and production systems produce huge amounts of data in various models, used for process monitoring, predictive maintenance of the machines, historical analysis and statistics, and more. Apache Hadoop brings a cost-effective opportunity for Big Data analysis, including the data generated in various industries. Integrating Hadoop into industrial environments creates new possibilities, as well as many challenges. The authors of this paper are involved into commercial and scientific projects utilizing Hadoop for industry as predictive analytics platform. In such initiatives the lack of standardization of monitoring of the industrial process in terms of Hadoop cluster utilization is especially perplexing. In this paper, authors propose the methodology of monitoring Hadoop in industrial environments, based on dedicated ontology and widely adopted standards: OPC Unified Architecture and RESTful Web Services.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Gartner. Big Data definition. Gartner IT glossary. http://www.gartner.com/it-glossary/big-data/. Accessed 28 Jan 2015

  2. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  3. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST). IEEE (2010)

    Google Scholar 

  4. Bakshi, K.: Considerations for big data: architecture and approach. In: Aerospace Conference, 2012. IEEE (2012)

    Google Scholar 

  5. White, T.: Hadoop: The Definitive Guide. O’Reilly Media Inc, Sebastopol (2012)

    Google Scholar 

  6. Bahga, A., Madisetti, V.K.: Analyzing massive machine maintenance data in a computing cloud. IEEE Trans. Parallel Distrib. Syst. 23(10), 1831–1843 (2012)

    Article  Google Scholar 

  7. Kiss, I., Genge, B., Haller, P., Sebestyen, G.: Data clustering-based anomaly detection in industrial control systems. In: 2014 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP). IEEE (2014)

    Google Scholar 

  8. Scholten, B.: The Road to Integration: A Guide to Applying the ISA-95 Standard in Manufacturing. Isa, USA (2007)

    Google Scholar 

  9. EMC2-Factory. http://www.emc2-factory.eu. Accessed 28 Jan 2015

  10. Factories of the Future. http://ec.europa.eu/research/industrial_technologies/factories-of-the-future_en.html. Accessed 28 Jan 2015

  11. Cupek, R., Drewniak, M., Zonenberg, D.: Online energy efficiency assessment in serial production-statistical and data mining approaches. In: 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE). IEEE (2014)

    Google Scholar 

  12. Cupek, R., Fojcik, M., Sande, O.: Object oriented vertical communication in distributed industrial systems. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2009. CCIS, vol. 39, pp. 72–78. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  13. Haaland Thorsen, K.A., Rong, C.: Towards dataintegration from WITSML to ISO 15926. In: Sandnes, F.E., Zhang, Y., Rong, C., Yang, L.T., Ma, J. (eds.) UIC 2008. LNCS, vol. 5061, pp. 626–635. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  14. King, R.L.: Information services for smart grids. In: Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy in the 21st Century, pp. 1–5. IEEE (2008)

    Google Scholar 

  15. Van Deursen, D., Poppe, C., Martens, G., Mannens, E., Walle, R.: XML to RDF conversion: a generic approach. In: 2008 International Conference on Automated Solutions for Cross Media Content and Multi-channel Distribution, AXMEDIS 2008. IEEE (2008)

    Google Scholar 

  16. Stopper, M., Katalinic, B.: Service-oriented architecture design aspects of OPC UA for industrial applications. In: Proceedings of the International Multi-Conference of Engineers and Computer Scientists (2009)

    Google Scholar 

  17. Rohjans, S., Fensel, D., Fensel, A.: OPC UA goes semantics: Integrated communications in smart grids. In: 2011 IEEE 16th Conference on Emerging Technologies and Factory Automation (ETFA), pp. 1–4 (2011)

    Google Scholar 

  18. Thorsen, K.A.H., Torbjørnse, O.F., Rong, C.: Automatic web service detection in oil and gas. In: Ślęzak, D., Kim, T., Chang, A.C.-C., Vasilakos, T., Li, M.C., Sakurai, K. (eds.) FGCN/ACN 2009. CCIS, vol. 56, pp. 193–200. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  19. Hortonworks Data Platform. http://hortonworks.com/hdp. Accessed 29 Jan 2015

  20. Richardson, L., Ruby, S.: RESTful Web Services. O’Reilly Media Inc, USA (2008)

    Google Scholar 

  21. Clavel, M., Durán, F., Eker, S., Lincoln, P., Martí-Oliet, N., Meseguer, J., Talcott, C.: Introduction to OPC UA performance. In: Clavel, M., Durán, F., Eker, S., Lincoln, P., Martí-Oliet, N., Meseguer, J., Talcott, C. (eds.) CN 2012, CCIS 291. LNCS, vol. 4350, pp. 1–28. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  22. Folkert, K., Fojcik, M., Cupek, R.: Efficiency of OPC UA communication in java-based implementations. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2011. CCIS, vol. 160, pp. 348–357. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

Download references

Acknowledgement

This work was supported by the European Union from the European Social Fund (grant agreement number: UDA-POKL.04.01.01-00-106/09).

Author information

Authors and Affiliations

  1. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Kamil Folkert

  2. Sogn Og Fjordane University College, Førde, Norway

    Marcin Fojcik

Authors
  1. Kamil Folkert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcin Fojcik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcin Fojcik .

Editor information

Editors and Affiliations

  1. Silesian University of Technology, Gliwice, Poland

    Piotr Gaj

  2. Silesian University of Technology, Gliwice, Poland

    Andrzej Kwiecień

  3. Silesian University of Technology, Gliwice, Poland

    Piotr Stera

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Folkert, K., Fojcik, M. (2015). Ontology-Based Integrated Monitoring of Hadoop Clusters in Industrial Environments with OPC UA and RESTful Web Services. In: Gaj, P., Kwiecień, A., Stera, P. (eds) Computer Networks. CN 2015. Communications in Computer and Information Science, vol 522. Springer, Cham. https://doi.org/10.1007/978-3-319-19419-6_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-19419-6_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-19418-9

  • Online ISBN: 978-3-319-19419-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation