Heterogeneous Resource Selection for Arbitrary HPC Applications in the Cloud

  • Anca IordacheEmail author
  • Eliya Buyukkaya
  • Guillaume Pierre
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9038)


Cloud infrastructures offer a wide variety of resources to choose from. However, most cloud users ignore the potential benefits of dynamically choosing cloud resources among a wide variety of VM instance types with different configuration/cost tradeoffs. We propose to automate the choice of resources that should be assigned to arbitrary non-interactive applications. During the first executions of the application, the system tries various resource configurations and builds a custom performance model for this application. Thereafter, cloud users can specify their execution time or financial cost constraints, and let the system automatically select the resources which best satisfy this constraint.


Execution Time Pareto Frontier Public Cloud Cloud Infrastructure Cloud User 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Tang, W., Desai, N., Buettner, D., Lan, Z.: Job scheduling with adjusted runtime estimates on production supercomputers. Elsevier Journal of Parallel and Distributed Computing (2013)Google Scholar
  2. 2.
    Amazon Elastic MapReduce,
  3. 3.
  4. 4.
    Allan, R.: Survey of HPC performance modelling and prediction tools. Technical Report DL-TR-2010-006, Science and Technology Facilities Council (July 2009)Google Scholar
  5. 5.
    Pllana, S., Brandic, I., Benkner, S.: A survey of the state of the art in performance modeling and prediction of parallel and distributed computing systems. IJCIR 4(1) (2008)Google Scholar
  6. 6.
    Beach, T.H., Rana, O.F., Avis, N.J.: Integrating acceleration devices using CometCloud. In: Proc. ORMaCloud Workshop (June 2013)Google Scholar
  7. 7.
    Vasic, N., Novakovic, D., Miucin, S., Kostic, D., Bianchini, R.: DejaVu: Accelerating resource allocation in virtualized environments. In: Proc. ACM ASPLOS (March 2012)Google Scholar
  8. 8.
    Fernandez, H., Pierre, G., Kielmann, T.: Autoscaling Web applications in heterogeneous cloud infrastructures. In: Proc. IEEE IC2E (March 2014)Google Scholar
  9. 9.
    Dejun, J., Pierre, G., Chi, C.H.: EC2 performance analysis for resource provisioning of service-oriented applications. In: NFPSLAM-SOC (November 2009)Google Scholar
  10. 10.
    Dejun, J., Pierre, G., Chi, C.H.: Resource provisioning of Web applications in heterogeneous clouds. In: Proc. USENIX WebApps (June 2011)Google Scholar
  11. 11.
    Farley, B., Juels, A., Varadarajan, V., Ristenpart, T., Bowers, K.D., Swift, M.M.: More for your money: exploiting performance heterogeneity in public clouds. In: SOCC (2012)Google Scholar
  12. 12.
    Oprescu, A.M., Kielmann, T., Leahu, H.: Budget estimation and control for bag-of-tasks scheduling in clouds. Parallel Processing Letters 21(2) (June 2011)Google Scholar
  13. 13.
    Verma, A., Cherkasova, L., Campbell, R.H.: ARIA: automatic resource inference and allocation for mapred uce environments. In: Proc. ICAC (2011)Google Scholar
  14. 14.
    Tian, F., Chen, K.: Towards optimal resource provisioning for running MapReduce programs in public clouds. In: Proc. IEEE CLOUD (2011)Google Scholar
  15. 15.
  16. 16.
    CopperEgg: AWS sizing tool,
  17. 17. Simulated annealingGoogle Scholar
  18. 18.
    CGC: Reverse time migration,
  19. 19.
    Sikka, V., Färber, F., Lehner, W., Cha, S.K., Peh, T., Bornhövd, C.: Efficient transaction processing in SAP HANA database – the end of a column store myth. In: SIGMOD (2012)Google Scholar
  20. 20.
  21. 21.

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  • Anca Iordache
    • 1
    Email author
  • Eliya Buyukkaya
    • 1
  • Guillaume Pierre
    • 1
  1. 1.RennesFrance

Personalised recommendations