SiL: An Approach for Adjusting Applications to Heterogeneous Systems Under Perturbations

  • Ali MohammedEmail author
  • Florina M. Ciorba
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11339)


Scientific applications consist of large and computationally-intensive loops. Dynamic loop scheduling (DLS) techniques are used to load balance the execution of such applications. Load imbalance can be caused by variations in loop iteration execution times due to problem, algorithmic, or systemic characteristics (also perturbations). The following question motivates this work: “Given an application, a high-performance computing (HPC) system, and their characteristics and interplay, which DLS technique will achieve improved performance under unpredictable perturbations?” Existing work only considers perturbations caused by variations in the HPC system delivered computational speeds. However, perturbations in available network bandwidth or latency are inevitable on production HPC systems. Simulator in the loop (SiL) is introduced, herein, as a new control-theoretic inspired approach to dynamically select DLS techniques that improve the performance of applications on heterogeneous HPC systems under perturbations. The present work examines the performance of six applications on a heterogeneous system under all above system perturbations. The SiL proof of concept is evaluated using simulation. The performance results confirm the initial hypothesis that no single DLS technique can deliver best performance in all scenarios, whereas the SiL-based DLS selection achieved improved application performance in most experiments.


Performance Load balancing Loop scheduling Heterogeneous computing systems Perturbations Simulation Computationally-intensive applications Simulator-in-the-loop 



This work has been supported by the Swiss Platform for Advanced Scientific Computing (PASC) project SPH-EXA: Optimizing Smooth Particle Hydrodynamics for Exascale Computing and by the Swiss National Science Foundation in the context of the Multi-level Scheduling in Large Scale High Performance Computers (MLS) grant number 169123. The authors gratefully acknowledge Ahmed Eleliemy for sharing an initial implementation of the PSIA application.


  1. 1.
    Ali, S., Maciejewski, A.A., Siegel, H.J., Kim, J.K.: Measuring the robustness of a resource allocation. IEEE Trans. Parallel Distrib. Syst. 15(7), 630–641 (2004)CrossRefGoogle Scholar
  2. 2.
    Banicescu, I., Liu, Z.: Adaptive factoring: a dynamic scheduling method tuned to the rate of weight changes. In: Proceedings of the High Performance Computing Symposium, pp. 122–129 (2000)Google Scholar
  3. 3.
    Banicescu, I., Velusamy, V., Devaprasad, J.: On the scalability of dynamic scheduling scientific applications with adaptive weighted factoring. Cluster Comput. 6(3), 215–226 (2003)CrossRefGoogle Scholar
  4. 4.
    Boulmier, A., Banicescu, I., Ciorba, F.M., Abdennadher, N.: An autonomic approach for the selection of robust dynamic loop scheduling techniques. In: Proceedings of 16th International Symposium on Parallel and Distributed Computing, pp. 9–17 (2017)Google Scholar
  5. 5.
    Browne, S., Dongarra, J., Garner, N., Ho, G., Mucci, P.: A portable programming interface for performance evaluation on modern processors. Int. J. High Perform. Comput. Appl. 14(3), 189–204 (2000)CrossRefGoogle Scholar
  6. 6.
    Canon, L.C., Jeannot, E.: Evaluation and optimization of the robustness of DAG schedules in heterogeneous environments. IEEE Trans. Parallel Distrib. Syst. 21(4), 532–546 (2010)CrossRefGoogle Scholar
  7. 7.
    Cariño, R.L., Banicescu, I.: Dynamic load balancing with adaptive factoring methods in scientific applications. J. Supercomput. 44(1), 41–63 (2008)CrossRefGoogle Scholar
  8. 8.
    Casanova, H., Giersch, A., Legrand, A., Quinson, M., Suter, F.: Versatile, scalable, and accurate simulation of distributed applications and platforms. J. Parallel Distrib. Comput. 74(10), 2899–2917 (2014)CrossRefGoogle Scholar
  9. 9.
    Ciorba, F.M.: The importance and need for system monitoring and analysis in HPC operations and research. In: Proceedings of the 3rd bwHPC-Symposium: Heidelberg 2016, pp. 7–16. heiBOOKS (2017)Google Scholar
  10. 10.
    Eleliemy, A., Mohammed, A., Ciorba, F.M.: Efficient generation of parallel spin-images using dynamic loop scheduling. In: Proceedings of the 19th IEEE International Conference for High Performance Computing and Communications Workshops, pp. 34–41 (2017)Google Scholar
  11. 11.
    Flynn Hummel, S., Schmidt, J., Uma, R.N., Wein, J.: Load-sharing in heterogeneous systems via weighted factoring. In: Proceedings of the Annual ACM Symposium on Parallel Algorithms and Architectures, pp. 318–328. ACM (1996)Google Scholar
  12. 12.
    Flynn Hummel, S., Schonberg, E., Flynn, L.E.: Factoring: a method for scheduling parallel loops. Commun. ACM 35(8), 90–101 (1992)CrossRefGoogle Scholar
  13. 13.
    García-González, L.A., García-Jacas, C.R., Acevedo-Martínez, L., Trujillo-Rasúa, R.A., Roose, D.: Self-scheduling for a heterogeneous distributed platform. In: Proceedings of the International Conference on Parallel Computing, ParCo2017, pp. 232–241. IOS press (2018)Google Scholar
  14. 14.
    Kruskal, C.P., Weiss, A.: Allocating independent subtasks on parallel processors. IEEE Trans. Softw. Eng. SE-11(10), 1001–1016 (1985)CrossRefGoogle Scholar
  15. 15.
    Mehrotra, R., Banicescu, I., Srivastava, S., Abdelwahed, S.: A power-aware autonomic approach for performance management of scientific applications in a data center environment. In: Khan, S., Zomaya, A. (eds.) Handbook on Data Centers, pp. 163–189. Springer, New York (2015). Scholar
  16. 16.
    Mohammed, A., Eleliemy, A., Ciorba, F.M., Kasielke, F., Banicescu, I.: Experimental verification and analysis of dynamic loop scheduling in scientific applications. In: Proceedings of the 17th International Symposium on Parallel and Distributed Computing, p. 8 (2018)Google Scholar
  17. 17.
    Mohammed, A., Eleliemy, A., Ciorba, F.M.: Performance reproduction and prediction of selected dynamic loop scheduling experiments. In: Proceedings of the 2018 International Conference on High Performance Computing and Simulation, p. 8 (2018)Google Scholar
  18. 18.
    Polychronopoulos, C.D., Kuck, D.J.: Guided self-scheduling: a practical scheduling scheme for parallel supercomputers. IEEE Trans. Comput. 36(12), 1425–1439 (1987)CrossRefGoogle Scholar
  19. 19.
    Rawlings, J.B.: Tutorial: overview of model predictive control. IEEE Control Syst. 20(3), 38–52 (2000)CrossRefGoogle Scholar
  20. 20.
    Sukhija, N., Malone, B., Srivastava, S., Banicescu, I., Ciorba, F.M.: Portfolio-based selection of robust dynamic loop scheduling algorithms using machine learning. In: Proceedings of 2014 IEEE International Parallel and Distributed Processing Symposium Workshops, pp. 1638–1647 (2014)Google Scholar
  21. 21.
    Tang, P., Yew, P.C.: Processor self-scheduling for multiple-nested parallel loops. In: Proceedings of the International Conference on Parallel Processing, pp. 528–535 (1986)Google Scholar
  22. 22.
    Velho, P., Legrand, A.: Accuracy study and improvement of network simulation in the SimGrid framework. In: Proceedings of the 2nd International Conference on Simulation Tools and Techniques, p. 10 (2009)Google Scholar
  23. 23.
    Yang, Y., Casanova, H.: RUMR: robust scheduling for divisible workloads. In: Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing, pp. 114–123 (2003)Google Scholar
  24. 24.
    Zhang, Y., Voss, M., Rogers, E.S.: Runtime empirical selection of loop schedulers on hyperthreaded SMPs. In: Proceedings of the 19th International Parallel and Distributed Processing Symposium, p. 10 (2005)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Mathematics and Computer ScienceUniversity of BaselBaselSwitzerland

Personalised recommendations