Improving a Run Time Job Prediction Model for Distributed Computing Based on Two Level Predictions

  • Hazem Al-Najjar
  • S. S. N. AlhadyEmail author
  • Junita Mohammad Saleh
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 547)


Nowadays, distributed computing environment faces many difficulties because the number of submitted jobs is increasing dramatically. One of the most used method to serve the jobs is to find the accurate run time of the submitted jobs. This paper proposes a new job prediction method, to predict on jobs’ run time using two level prediction namely linear regression model and fitting model. The proposed model uses six variables including user ID, group ID, executable ID, number of CPUs, memory size and average CPU time, furthermore to solve the problem of the categorical variables (i.e. user ID, group ID and executable ID) a dummy code is used. To adjust and to find the best combination between linear regression model and fitting models, different fitting models are used by combining linear and nonlinear fitting models. By simulation the results show that the proposed model is better than previous models when smoothing spline fitting is used, also the results indicate that proposed model is efficient with low error and high prediction rate compared with previous models.


Job prediction Back propagation neural network Distributed computing Fitting model Linear regression model 


  1. 1.
    Nepovinnykh, E.A., Radchenko, G. I. (2016). Problem-oriented scheduling of cloud applications: PO-HEFT algorithm case study. In: 2016 39th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 180–185. IEEEGoogle Scholar
  2. 2.
    Attiya, I., Zhang, X., Yang, X.: TCSA: a dynamic job scheduling algorithm for computational grids. In: 2016 IEEE International Conference on Computer Communication and the Internet (ICCCI), pp. 408–412. IEEE (2016)Google Scholar
  3. 3.
    Yu, S., Kak, S.: A survey of prediction using social media (2012). arXiv:1203.1647
  4. 4.
    Dinda, P.A.: Online prediction of the running time of tasks. Clust. Comput. 5(3), 225–236 (2002)CrossRefGoogle Scholar
  5. 5.
    Smith, W.: Prediction services for distributed computing. In: IEEE International Parallel and Distributed Processing Symposium, 2007. IPDPS 2007, pp. 1–10. IEEE (2007)Google Scholar
  6. 6.
    Dimitriadou, S., Karatza, H.: Job scheduling in a distributed system using backfilling with inaccurate runtime computations. In: 2010 International Conference on Complex, Intelligent and Software Intensive Systems (CISIS), pp. 329–336. IEEE (2010)Google Scholar
  7. 7.
    Pietri, I., Juve, G., Deelman, E., Sakellariou, R.: A performance model to estimate execution time of scientific workflows on the cloud. In: Proceedings of the 9th Workshop on Workflows in Support of Large-Scale Science, pp. 11–19. IEEE Press (2014)Google Scholar
  8. 8.
    Cunha, R.L., Rodrigues, E.R., Tizzei, L.P., Netto, M.A.: Job placement advisor based on turnaround predictions for HPC hybrid clouds. Futur. Gener. Comput. Syst. 67, 35–46 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Hazem Al-Najjar
    • 1
  • S. S. N. Alhady
    • 1
    Email author
  • Junita Mohammad Saleh
    • 1
  1. 1.School of Electrical & Electronic EngineeringUniversiti Sains Malaysia (USM)Nibong Tebal, PenangMalaysia

Personalised recommendations