Skip to main content

Evaluating Energy-Aware Scheduling Algorithms for I/O-Intensive Scientific Workflows

  • Conference paper
  • First Online:
Computational Science – ICCS 2021 (ICCS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12742))

Included in the following conference series:

Abstract

Improving energy efficiency has become necessary to enable sustainable computational science. At the same time, scientific workflows are key in facilitating distributed computing in virtually all domain sciences. As data and computational requirements increase, I/O-intensive workflows have become prevalent. In this work, we evaluate the ability of twopopular energy-aware workflow scheduling algorithms to provide effective schedules for this class of workflow applications, that is, schedules that strike a good compromise between workflow execution time and energy consumption. These two algorithms make decisions based on a widely used power consumption model that simply assumes linear correlation to CPU usage. Previous work has shown this model to be inaccurate, in particular for modeling power consumption of I/O-intensive workflow executions, and has proposed an accurate model. We evaluate the effectiveness of the two aforementioned algorithms based on this accurate model. We find that, when making their decisions, these algorithms can underestimate power consumption by up to 360%, which makes it unclear how well these algorithm would fare in practice. To evaluate the benefit of using the more accurate power consumption model in practice, we propose a simple scheduling algorithm that relies on this model to balance the I/O load across the available compute resources. Experimental results show that this algorithm achieves more desirable compromises between energy consumption and workflow execution time than the two popular algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Casanova, H., et al.: Developing accurate and scalable simulators of production workflow management systems with wrench. Future Gener. Comp. Sy. 112, 162–175 (2020)

    Article  Google Scholar 

  2. Chameleon cloud. https://chameleoncloud.org (2021)

  3. De Oliveira, D.C., et al.: Data-intensive workflow management: for clouds and data-intensive and scalable computing environments. Synth. Lect. Data Manage. 14(4), 1–179 (2019)

    Article  Google Scholar 

  4. Deelman, E., et al.: Pegasus, a workflow management system for science automation. Futur. Gener. Comput. Syst. 46, 17–35 (2015)

    Article  Google Scholar 

  5. Ghose, M., et al.: Energy efficient scheduling of scientific workflows in cloud environment. In: IEEE HPCC (2017)

    Google Scholar 

  6. Klimentov, A., et al.: Next generation workload management system for big data on heterogeneous distributed computing. J. Phys: Conf. Ser. 608, 012040 (2015)

    Google Scholar 

  7. Li, Z., et al.: Cost and energy aware scheduling algorithm for scientific workflows with deadline constraint in clouds. IEEE Trans. Serv. Comput. 11(4), 713–726 (2018)

    Article  Google Scholar 

  8. Ma, X., et al.: An IoT-based task scheduling optimization scheme considering the deadline and cost-aware scientific workflow for cloud computing. EURASIP J. Wirel. Commun. Netw. 2019(1), 1–19 (2019)

    Article  Google Scholar 

  9. Orgerie, A.C., et al.: A survey on techniques for improving the energy efficiency of large-scale distributed systems. ACM Comput. Surv. (CSUR) 46(4), 1–31 (2014)

    Article  Google Scholar 

  10. Pietri, I., Sakellariou, R.: Energy-aware workflow scheduling using frequency scaling. In: International Conference on Parallel Processing Workshops (2014)

    Google Scholar 

  11. Pietri, I., et al.: Energy-constrained provisioning for scientific workflow ensembles. In: International Conference on Cloud and Green Computing (CGC) (2013)

    Google Scholar 

  12. Qin, Y., Wang, H., Yi, S., Li, X., Zhai, L.: An energy-aware scheduling algorithm for budget-constrained scientific workflows based on multi-objective reinforcement learning. J. Supercomput. 76(1), 455–480 (2019). https://doi.org/10.1007/s11227-019-03033-y

    Article  Google Scholar 

  13. Shepherd, D., et al.: Workflow scheduling on power constrained VMs. In: IEEE/ACM 8th International Conference on Utility and Cloud Computing (2015)

    Google Scholar 

  14. Ferreira da Silva, R., et al.: A characterization of workflow management systems for extreme-scale applications. Future Gener. Comput. Syst. 75, 228–238 (2017)

    Google Scholar 

  15. Ferreira da Silva, R., Orgerie, A.-C., Casanova, H., Tanaka, R., Deelman, E., Suter, F.: Accurately simulating energy consumption of I/O-intensive scientific workflows. In: Rodrigues, J., et al. (eds.) ICCS 2019. LNCS, vol. 11536, pp. 138–152. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22734-0_11

    Chapter  Google Scholar 

  16. Ferreira da Silva, R., et al.: Using simple pid-inspired controllers for online resilient resource management of distributed scientific workflows. Future Gener. Comp. Sy. 95 (2019)

    Google Scholar 

  17. Ferreira da Silva, R., et al.: Characterizing, modeling, and accurately simulating power and energy consumption of i/o-intensive scientific workflows. Journal of Computational Science 44, 101157 (2020)

    Google Scholar 

  18. Ferreira da Silva, R., et al.: Workflowhub: Community framework for enabling scientific workflow research and development. In: IEEE WORKS Workshop (2020)

    Google Scholar 

  19. Energy-aware simulator. https://github.com/wrench-project/energy-aware-simulator (2021)

  20. Wang, X., et al.: Delay-cost tradeoff for virtual machine migration in cloud data centers. J. Netw. Comput. Appl. 78, 62–72 (2017)

    Article  Google Scholar 

  21. Wu, T., et al.: Soft error-aware energy-efficient task scheduling for workflow applications in DVFS-enabled cloud. J. Syst. Architect. 84, 12–27 (2018)

    Article  Google Scholar 

  22. Xu, X., et al.: EnReal: an energy-aware resource allocation method for scientific workflow executions in cloud environment. IEEE Trans. Cloud Comput. 4(2), 166–179 (2015)

    Google Scholar 

Download references

Acknowledgments

This work is funded by NSF contracts #1923539, #1923621, #2016610, and #2016619. Results presented in this paper were obtained using the Chameleon testbed supported by the National Science Foundation.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Tainã Coleman or Rafael Ferreira da Silva .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Coleman, T., Casanova, H., Gwartney, T., da Silva, R.F. (2021). Evaluating Energy-Aware Scheduling Algorithms for I/O-Intensive Scientific Workflows. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2021. ICCS 2021. Lecture Notes in Computer Science(), vol 12742. Springer, Cham. https://doi.org/10.1007/978-3-030-77961-0_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77961-0_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77960-3

  • Online ISBN: 978-3-030-77961-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics