Advertisement

Toward Smarter Hadoop’s Slaves Nodes by Deploying Game Theory Strategies

  • Ahmed Qasim MohammedEmail author
  • Aman Singh
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 34)

Abstract

As each one of us recognized the high speed of evaluation in the technologies field, the result of which is generating a huge amount of data which is known as Big Data. For this Big Data, there is need for an efficient distributed system to process this data and here came the Hadoop. Hadoop is one of the important frameworks in distributed system that has many other applications to process data. There are many researchers working to improve the performance of Hadoop, but in all literature the main focus is to improve the Master node, while it is important to try and provide a smarter Slaves node that can cooperate or predict what other Slaves node strategies to pick up the Job. In this paper, our study focuses on Game theory domain such as Nash equilibrium and bargaining strategy, second is Artificial intelligent to propose a smarter system even slave nodes in it can take a decision, especially that the core of Hadoop System is adopting a pull-scheduling strategy. We believe our work is going to improve resources utilization and minimize the processing time of Jobs.

Keywords

Hadoop Nash equilibrium Bargaining strategy Big data Artificial intelligent Resource utilization 

References

  1. 1.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  2. 2.
    White, T.: How the MapReduce works, Hadoop: The Definitive Guide, 3rd edn. O’Reilly Inc., Tokyo, Japan (2012)Google Scholar
  3. 3.
    Borthakur, D.: The hadoop distributed file system: architecture and design. Hadoop Project Website 11(2007), 21 (2007)Google Scholar
  4. 4.
    deRoos, D.: Hadoop for Dummies. A Wiley Brand (2014)Google Scholar
  5. 5.
    Lam, C.: Hadoop in Action. Manning Publications Co. http://hadoop.apache.org (2010)
  6. 6.
  7. 7.
    Nash Jr., J.F.: The bargaining problem. Econometr. J. the Econometr. Soc., 155–162 (1950)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Zaharia, M., Konwinski, A., Joseph, A.D., Katz, R., Stoica, I.: Improving MapReduce performance in heterogeneous environments. In: OSDI 2008, pp. 29–42. USENIX Association, Berkeley (2008)Google Scholar
  9. 9.
    Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A.: Quincy: fair scheduling for distributed computing clusters. In: Proceedings of the ACM SIGOPS 22nd symposium on operating systems principles, pp. 261–276. ACM (2009)Google Scholar
  10. 10.
    Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: distributed data-parallel programs from sequential building blocks. In: ACM SIGOPS Operating Systems Review, vol. 41, No. 3, pp. 59–72. ACM (2007)CrossRefGoogle Scholar
  11. 11.
    Zhang, Q., Zhani, M., Yang, Y., Boutaba, R., Wong, B.: PRISM: fine-grained resource-aware scheduling for MapReduce. IEEE Trans. Cloud Comput. 3, 182–194 (2015)CrossRefGoogle Scholar
  12. 12.
    Tang, S., Lee, B.S., He, B.: Dynamic job ordering and slot configurations for MapReduce workloads. IEEE Trans. Serv. Comput. 9, 4–17 (2016)CrossRefGoogle Scholar
  13. 13.
    Polo, J., Becerra, Y., Carrera, D., Steinder, M.: Deadline-based MapReduce workload management. IEEE Trans. Netw. Serv. Manag. 10, 231–244 (2013)Google Scholar
  14. 14.
    Tang, S., Lee, B.S., He, B.: MROrder: Flexible Job Ordering Optimization for Online MapReduce Workloads, vol. 8097, pp. 291–304 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Savitribai Phule Pune UniversityPuneIndia

Personalised recommendations