Skip to main content
SpringerLink
Log in

MapReduce Hadoop Models for Distributed Neural Network Processing of Big Data Using Cloud Services

  • Conference paper
  • First Online:
Advances in Intelligent Systems and Computing IV (CSIT 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1080))

Included in the following conference series:

  • 652 Accesses

Abstract

The paper proposes a formalization process of Big Data distributed intelligent processing using Cloud-Fog-Dew architecture. This process provides specialized services, based on continuous support of experts in areas of concern, and advisory support for their actions in diagnostically complex cases. The method of Big Data processing based on neural networks is considered, which is distinguished by dynamic redistribution of work between computers, that allows to uniformly load the computing cluster with different data topologies. Proposed method is less by one order of computational complexity and less time spent. The MapReduce Hadoop models for distributed neural network processing of Big Data were proposed, characterized by the adaptation of data topology to the corresponding architectural computer cluster. This reduces the amount of information transmitted between nodes to increase productivity in solving complex tasks and effectively balancing the load of computing resources with different data topologies. An experimental Hadoop cluster was created to evaluate the performance of developed models for Big Data distributed processing. It allows for the implementation of parallel learning procedures for multilayer neural networks based on “star” and “fully connected graph” data topology with different amounts of input data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Marjani, M.: Big IoT data analytics: architecture, opportunities, and open research chasllenges. IEEE Access 5, 5247–5261 (2017)

    Article  Google Scholar 

  2. Nikolaychuk, Ya.M.: Specialized computer technologies in informatics: monography. In: Nikolaychuk, Ya.M. (ed.) Beskydy, Ternopil, 919 p. (2017). (in ukrainian)

    Google Scholar 

  3. Haykin, S.: Neural Networks and Learning Machines, 3rd edn., 937 p. Pearson Education (2009)

    Google Scholar 

  4. Bogdanova, V.G.: Multiagent approach to controlling distributed computing in a cluster grid system. J. Comput. Syst. Sci. Int. 53, 713–722 (2014)

    Article  Google Scholar 

  5. Lützenberger, M.: Multi-agent system in practice: when research meets reality. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, pp. 796–805 (2016)

    Google Scholar 

  6. Fernandes, L.M., O’Connor, M., Weaver, V.: Big data, bigger outcomes. J. AHIMA 83(10), 38–43 (2012)

    Google Scholar 

  7. Patibandla, R.S.M.L., Veeranjaneyulu, N.: Survey on clustering algorithms for unstructured data. In: Intelligent Engineering Informatics, pp. 421–429. Springer, Singapore (2018)

    Google Scholar 

  8. Shirkhorshidi, A.S.: Big data clustering: a review. In: International Conference on Computational Science and Its Applications, pp. 707–720. Springer, Cham (2014)

    Google Scholar 

  9. Chen, H.: Smart health and wellbeing [Trends & Controversies]. IEEE Intell. Syst. 26(5), 78–90 (2011)

    Article  Google Scholar 

  10. Axak, N., Rosinskiy, D., Barkovska O., Novoseltsev, I.: Cloud-fog-dew architecture for personalized service-oriented systems. In: The 9th IEEE International Conference on Dependable Systems, Services and Technologies, DESSERT 2018, Kyiv, pp. 80–84 (2018)

    Google Scholar 

  11. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  12. Ghemawat, S., Gobioff, H., Leung, ST.: The Googlefile system. SIGOPS Oper. Syst. Rev. 37, 29–43 (2003)

    Google Scholar 

  13. Axak, N.G., Lebyodkina, A.Yu.: Method of uniform distribution of parallel operatrions for the multy-layer neural network accellerated learning based on different data topologies. Control. Navig. Commun. Syst. 2(18), 66–73 (2011). (in Russian)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kharkiv National University of Radio Electronics, Nauky Ave. 14, Kharkiv, 61166, Ukraine

    Natalia Axak, Mykola Korablyov & Dmytro Rosinskiy

Authors
  1. Natalia Axak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mykola Korablyov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dmytro Rosinskiy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mykola Korablyov .

Editor information

Editors and Affiliations

  1. Department of Artificial Intelligence, Lviv Polytechnic National University, Lviv, Ukraine

    Natalya Shakhovska

  2. Institute of Computer Science and Information Technologies, Lviv Polytechnic National University, Lviv, Ukraine

    Mykola O. Medykovskyy

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Axak, N., Korablyov, M., Rosinskiy, D. (2020). MapReduce Hadoop Models for Distributed Neural Network Processing of Big Data Using Cloud Services. In: Shakhovska, N., Medykovskyy, M.O. (eds) Advances in Intelligent Systems and Computing IV. CSIT 2019. Advances in Intelligent Systems and Computing, vol 1080. Springer, Cham. https://doi.org/10.1007/978-3-030-33695-0_27

Download citation

Publish with us

Policies and ethics

Search

Navigation