Skip to main content
SpringerLink
Log in

Heterogeneous Exascale Computing

  • Chapter
  • First Online:
Recent Advances in Intelligent Engineering

Part of the book series: Topics in Intelligent Engineering and Informatics ((TIEI,volume 14))

Abstract

Exascale services bring new unique challenges that the current computational, big data and workflow solutions are unable to meet. The chapter includes a detailed description of selected exascale services with known state of the art in extreme date solutions. The integration of requirements and the analysis of the state of the art in the exascale field is centered in on a description of a high-level architectural approach. The next main contribution of the paper is the description of the architecture capable to handle heterogeneous exascale services coming from both academic as well as industrial sphere. Those two models represent a (conceptual, and technological) design of a platform that addresses the requirements of the use cases. The resulting architecture will help us provide computing solutions to exascale challenges within the H2020 project PROCESS.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    PROCESS project homepage https://www.process-project.eu/.

  2. 2.

    http://www.lofar.org/wiki/doku.php?id=public:grid_srm_software_installation.

  3. 3.

    http://www.lofar.org/wiki/doku.php?id=public:grid_srm_software_installation.

  4. 4.

    http://www.lofar.org/wiki/doku.php?id=public:grid_srm_software_installation.

  5. 5.

    https://cernvm.cern.ch/portal/filesystem.

  6. 6.

    http://docs.surfsaralabs.nl/projects/grid/en/latest/Pages/Practices/picas/picas_overview.html.

  7. 7.

    https://github.com/lofar-astron/prefactor.

  8. 8.

    https://github.com/mhardcastle/ddf-pipeline.

  9. 9.

    The subchapter provides an example of a technology-based architecture that meets the requirements of the use cases (presented above) as well as the conceptual and functional requirements derived from the functional model of the PROCESS architecture.

  10. 10.

    Details concerning the MEE can be found at http://www.cyfronet.krakow.pl/cgw17/presentations/S7_2-EurValve-MEE-Oct-2017-v02.pdf.

  11. 11.

    Julpiter webpage http://jupyter.org/.

  12. 12.

    LOBCDER2] LOBCDER webpage https://ivi.fnwi.uva.nl/sne/wsvlam2/?page_id=14.

  13. 13.

    OASIS Topology and Orchestration Specification for https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca.

  14. 14.

    Cloudify] Cloudify: Cloud & NFV Orchestration Based on TOSCA https://cloudify.co/.

  15. 15.

    RIMROCK webpage https://submit.plgrid.pl.

  16. 16.

    Atmosphere webpage http://dice.cyfronet.pl/products/atmosphere.

  17. 17.

    Atkinson, M., Brezany, P., Krause, A., van Hemert, J., Janciak, I., Yaikhom, G.: DISPEL: Grammar and Concrete Syntax, version 1.0. The Admire Project, February 2010. Accessed March 2011. http://www.Admire-project.eu/docs/Admire-D1.7-research-prototypes.pdf.

References

  1. R. Agarwal, G. Juve, E. Deelman, Peer-to-peer data sharing for scientific workflows on amazon ec2, in 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC) (IEEE, 2012), pp. 82–89

    Google Scholar 

  2. B. Allcock, I. Mandrichenko, T. Perelmutov, Gridftp v2 protocol description. GridFTP Working Group, Technical report, 2005

    Google Scholar 

  3. S. Ashby, P. Beckman, J. Chen, P. Colella, B. Collins, D. Crawford, J. Dongarra, D. Kothe, R. Lusk, P. Messina, et al., The opportunities and challenges of exascale computing–summary report of the advanced scientific computing advisory committee (ASCAC) subcommittee. US Department of Energy Office of Science (2010)

    Google Scholar 

  4. M. Bobák, A.S.Z. Belloum, P. Nowakowski, J. Meizner, M. Bubak, M. Heikkurinen, O. Habala, L. Hluchý, Exascale computing and data architectures for brownfield applications, in 2018 14th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD) (IEEE, 2018), pp. 461–468

    Google Scholar 

  5. J. Chen, A. Choudhary, S. Feldman, B. Hendrickson, C. Johnson, R. Mount, V. Sarkar, V. White, D. Williams, Synergistic challenges in data-intensive science and exascale computing: doe ASCAC data subcommittee report (2013)

    Google Scholar 

  6. L.B. Costa, H. Yang, E. Vairavanathan, A. Barros, K. Maheshwari, G. Fedak, D. Katz, M. Wilde, M. Ripeanu, S. Al-Kiswany, The case for workflow-aware storage: an opportunity study. J. Grid Comput. 13(1), 95–113 (2015)

    Article  Google Scholar 

  7. F. Dottori, P. Salamon, A. Bianchi, L. Alfieri, F.A. Hirpa, L. Feyen, Development and evaluation of a framework for global flood hazard mapping. Adv. Water Resour. 94, 87–102 (2016)

    Article  Google Scholar 

  8. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778

    Google Scholar 

  9. O. Jimenez-del Toro, S. Otálora, M. Andersson, K. Eurén, M. Hedlund, M. Rousson, H. Müller, M. Atzori, Analysis of histopathology images: from traditional machine learning to deep learning, in Biomedical Texture Analysis (Elsevier, Amsterdam, 2018), pp. 281–314

    Google Scholar 

  10. M. Kluge, S. Simms, T. William, R. Henschel, A. Georgi, C. Meyer, M.S. Mueller, C.A. Stewart, W. Wünsch, W.E. Nagel, Performance and quality of service of data and video movement over a 100 gbps testbed. Futur. Gener. Comput. Syst. 29(1), 230–240 (2013)

    Article  Google Scholar 

  11. D. Kreutz, F.M. Ramos, P.E. Verissimo, C.E. Rothenberg, S. Azodolmolky, S. Uhlig, Software-defined networking: a comprehensive survey. Proc. IEEE 103(1), 14–76 (2015)

    Article  Google Scholar 

  12. Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan, C. Zhang, Learning efficient convolutional networks through network slimming, in 2017 IEEE International Conference on Computer Vision (ICCV) (IEEE, 2017), pp. 2755–2763

    Google Scholar 

  13. G. Montavon, W. Samek, K.-R. Müller, Methods for interpreting and understanding deep neural networks. Digit. Signal Process. (2017)

    Google Scholar 

  14. R. Rudari, F. Silvestro, L. Campo, N. Rebora, G. Boni, CIMA Research Foundation, C. Herold, UNEP-GRID, Improvement of the global food model for the gar 2015. Global Assessment Report on Disaster Risk Reduction 2015 (2015)

    Google Scholar 

  15. T. Shimwell, H. Röttgering, P.N. Best, W. Williams, T. Dijkema, F. De Gasperin, M. Hardcastle, G. Heald, D. Hoang, A. Horneffer et al., The lofar two-metre sky survey-i. Survey description and preliminary data release. Astron. Astrophys. 598, A104 (2017)

    Article  Google Scholar 

  16. A. Sim, A. Shoshani, The storage resource manager interface specification, version 2.2, in CERN, FNAL, JLAB, LBNL and RAL (Citeseerx, 2007)

    Google Scholar 

  17. A. Simonet, G. Fedak, M. Ripeanu, Active data: a programming model to manage data life cycle across heterogeneous systems and infrastructures. Futur. Gener. Comput. Syst. 53, 25–42 (2015)

    Article  Google Scholar 

  18. Storage Networking Industry Association. Cloud data management interface (CDMI) (2010)

    Google Scholar 

  19. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in Proceedings of the IEEE conference on computer vision and pattern recognition (2015), pp. 1–9

    Google Scholar 

  20. R. Tudoran, A. Costan, O. Nano, I. Santos, H. Soncu, G. Antoniu, Jetstream: enabling high throughput live event streaming on multi-site clouds. Futur. Gener. Comput. Syst. 54, 274–291 (2016)

    Article  Google Scholar 

  21. L. Wang, J. Tao, R. Ranjan, H. Marten, A. Streit, J. Chen, D. Chen, G-hadoop: mapreduce across distributed data centers for data-intensive computing. Futur. Gener. Comput. Syst. 29(3), 739–750 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by projects EU H2020-777533 PROCESS PROviding Computing solutions for ExaScale ChallengeS, APVV-17-0619, and VEGA 2/0167/16.

Author information

Authors and Affiliations

  1. Institute of Informatics, Slovak Academy of Sciences, Dúbravská cesta 9, SK-845 07, Bratislava, Slovakia

    Ladislav Hluchý, Martin Bobák, Ondrej Habala, Martin Šeleng, Štefan Dlugolinský, Viet Tran & Giang Nguyen

  2. University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland

    Henning Müller, Mara Graziani & Matti Heikkurinen

  3. Netherlands eScience Center, Science Park 123, Amsterdam, Netherlands

    Jason Maassen & Hanno Spreeuw

  4. Lufthansa Systems, Siemensdamm 62, D-13627, Berlin, Germany

    Jörg Pancake-Steeg & Stefan Spahr

  5. Ludwig-Maximilians Universitaet, Munich, Germany

    Nils Otto vor dem Gentschen Felde, Maximilian Höb & Jan Schmidt

  6. Informatics Institute, University of Amsterdam, P.O. Box 94323, 1090 GH, Amsterdam, Netherlands

    Adam S. Z. Belloum & Reginald Cushing

  7. ACC Cyfronet AGH, AGH University of Science and Technology, ul. Nawojki 11, 30-950, Krakow, Poland

    Piotr Nowakowski, Jan Meizner, Katarzyna Rycerz, Bartosz Wilk & Marian Bubak

Authors
  1. Ladislav Hluchý
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Bobák
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Henning Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mara Graziani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jason Maassen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hanno Spreeuw
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Matti Heikkurinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jörg Pancake-Steeg
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Stefan Spahr
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Nils Otto vor dem Gentschen Felde
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Maximilian Höb
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Jan Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Adam S. Z. Belloum
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Reginald Cushing
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Piotr Nowakowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Jan Meizner
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Katarzyna Rycerz
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Bartosz Wilk
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Marian Bubak
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Ondrej Habala
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Martin Šeleng
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Štefan Dlugolinský
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Viet Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Giang Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ladislav Hluchý .

Editor information

Editors and Affiliations

  1. Obuda University, Budapest, Hungary

    Levente Kovács

  2. Óbuda University, Budapest, Hungary

    Tamás Haidegger

  3. Óbuda University, Budapest, Hungary

    Anikó Szakál

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Hluchý, L. et al. (2020). Heterogeneous Exascale Computing. In: Kovács, L., Haidegger, T., Szakál, A. (eds) Recent Advances in Intelligent Engineering. Topics in Intelligent Engineering and Informatics, vol 14. Springer, Cham. https://doi.org/10.1007/978-3-030-14350-3_5

Download citation

Publish with us

Policies and ethics

Search

Navigation