Adaptive DAG Tasks Scheduling with Deep Reinforcement Learning

  • Qing Wu
  • Zhiwei Wu
  • Yuehui Zhuang
  • Yuxia ChengEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11335)


Efficient task scheduling is critical for improving system performance in the distributed heterogeneous computing environment. The DAG (Directed Acyclic Graph) tasks scheduling problem is NP-complete and it is hard to find an optimal schedule. Due to its key importance, the DAG tasks scheduling problem has been extensively studied in the literature. Many previously proposed heuristic algorithms are usually based on greedy methods, which still exists large optimization space to be explored. In this paper, we proposed an adaptive DAG tasks scheduling (ADTS) algorithm using deep reinforcement learning. The scheduling problem is properly defined with the reinforcement learning process. Efficient scheduling state space, action space and reward function are designed to train the policy gradient-based REINFORCE agent. Leveraging the algorithm’s capability of exploring long term reward, the ADTS algorithm could achieve good scheduling policies. Experimental results showed the effectiveness of the proposed ADTS algorithm compared with the classic HEFT/CPOP algorithms.


DAG scheduling Heterogeneous Deep reinforcement learning 


  1. 1.
    Abadi, M., et al.: TensorFlow: a system for large-scale machine learning. OSDI 16, 265–283 (2016)Google Scholar
  2. 2.
    Ahmad, I., Kwok, Y.K.: On exploiting task duplication in parallel program scheduling. IEEE Trans. Parallel Distrib. Syst. 9(9), 872–892 (1998)CrossRefGoogle Scholar
  3. 3.
    Amalarethinam, D., Josphin, A.M.: Dynamic task scheduling methods in heterogeneous systems: a survey. Int. J. Comput. Appl. 110(6), 12–18 (2015)Google Scholar
  4. 4.
    Arabnejad, H., Barbosa, J.G.: List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Trans. Parallel Distrib. Syst. 25(3), 682–694 (2014)CrossRefGoogle Scholar
  5. 5.
    Browne, C.B., et al.: A survey of monte carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)CrossRefGoogle Scholar
  6. 6.
    Goldie, A., Mirhoseini, A., Steiner, B., Pham, H., Dean, J., Le, Q.V.: Hierarchical planning for device placement. In: Proceedings of ICLR, pp. 1–11 (2018)Google Scholar
  7. 7.
    Kanemitsu, H., Hanada, M., Nakazato, H.: Clustering-based task scheduling in a large number of heterogeneous processors. IEEE Trans. Parallel Distrib. Syst. 27(11), 3144–3157 (2016)CrossRefGoogle Scholar
  8. 8.
    Kwok, Y.K., Ahmad, I.: Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Comput. Surv. 31(4), 406–471 (1999)CrossRefGoogle Scholar
  9. 9.
    Mao, H., Alizadeh, M., Menache, I., Kandula, S.: Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM Workshop on Hot Topics in Networks, pp. 50–56. ACM (2016)Google Scholar
  10. 10.
    Mayer, R., Mayer, C., Laich, L.: The TensorFlow partitioning and scheduling problem: it’s the critical path! In: Proceedings of the 1st Workshop on Distributed Infrastructures for Deep Learning, pp. 1–6. ACM (2017)Google Scholar
  11. 11.
    Mirhoseini, A., et al.: Device placement optimization with reinforcement learning. In: Proceedings of ICML, pp. 2430–2439 (2017)Google Scholar
  12. 12.
    Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937 (2016)Google Scholar
  13. 13.
    Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)CrossRefGoogle Scholar
  14. 14.
    Orhean, A.I., Pop, F., Raicu, I.: New scheduling approach using reinforcement learning for heterogeneous distributed systems. J. Parallel Distrib. Comput. (2017)Google Scholar
  15. 15.
    Palis, M.A., Liou, J.C., Wei, D.S.L.: Task clustering and scheduling for distributed memory parallel architectures. IEEE Trans. Parallel Distrib. Syst. 7(1), 46–55 (1996)CrossRefGoogle Scholar
  16. 16.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2011)zbMATHGoogle Scholar
  17. 17.
    Topcuoglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)CrossRefGoogle Scholar
  18. 18.
    Ullman, J.D.: NP-complete scheduling problems. J. Comput. Syst. Sci. 10(3), 384–393 (1975)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Wu, A.S., Yu, H., Jin, S., Lin, K.C., Schiavone, G.: An incremental genetic algorithm approach to multiprocessor scheduling. IEEE Trans. Parallel Distrib. Syst. 15(9), 824–834 (2004)CrossRefGoogle Scholar
  20. 20.
    Xian-Fu, M., Wei-Wei, L.: A DAG scheduling algorithm based on selected duplication of precedent tasks. J. Comput.-Aided Des. Comput. Graph. 6, 023 (2010)Google Scholar
  21. 21.
    Xu, Y., Li, K., Hu, J., Li, K.: A genetic algorithm for task scheduling on heterogeneous computing systems using multiple priority queues. Inf. Sci. 270, 255–287 (2014)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Zhang, W., Dietterich, T.G.: A reinforcement learning approach to job-shop scheduling. IJCAI 95, 1114–1120 (1995)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Qing Wu
    • 1
  • Zhiwei Wu
    • 1
  • Yuehui Zhuang
    • 2
  • Yuxia Cheng
    • 1
    Email author
  1. 1.Hangzhou Dianzi UniversityHangzhouChina
  2. 2.Zhejiang Fangzheng Media Technology Research InstituteHangzhouChina

Personalised recommendations