Advertisement

Reduction of Traffic on Roads Using Big Data Applications

  • P. AjithaEmail author
  • A. Sivasangari
  • G. Lakshmi Mounica
  • Lakshmi Prathyusha
Conference paper
  • 41 Downloads
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 49)

Abstract

Mapreduce implementation simplifies big amount of data implementation on complex and large datasets by using parallelization of map tasks and reduce tasks. Several contributions and methodologies are made to eradicate the functionality and usage of Mapreduce tasks, the data generated in the shuffle phase should be ignored, which contributes as an important feature in functionality development. Shuffle phase uses hash aggregator to merge information into clusters which is useless in managing traffic data creating a bottleneck. Improvisation of the execution of system traffic in the shuffle stage is fundamental to enhance the execution of the tasks. Main objective of slacking off the structured dataset is developed using utilisation tasks and total. The implemented course of action is known to limit and sort out traffic cost in Mapreduce. The hash aggregators are implemented in various tasks, where every aggregator can diminish overall data traffic from different tasks. This dispersed estimation is supposed to maintain the extensive data organizing problem for huge data implemented applications. Besides, an integrated computation is proposed to change the data and assemble successfully in a proper orientation. In this paper, a study of traffic cost for a Map reduce tasks by determining new intermediate data division schema is carried out. We consider data lifting and other applications that have all around attributes from past data that is collected bit-by-bit using Map reduce. For example, a count that has all the means of taking after a word checking at the key signal may wrap up having all around various attributes, for instance, the numbers and transitional results, in that limit, we recognize the startling execution. Grabbing the data from various applications, we review the detailed report for bolstering data focused and implement the new data applications on server ranch scale systems. The promising results may include improving a business control structure with a medley of connections and function that support in a Map reduce structure. Finally, the outcome exhibits that our manifesto can tremendously diminish traffic congestion beneath any conditions.

Keywords

Map reduce Cluster Aggregator 

References

  1. 1.
    Deanand, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  2. 2.
    Wang, W., Zhu, K., Ying, L., Tan, J., Zhang, L.: Maptaskscheduling InMapReduce with data locality: throughput and heavy-traffic optimality. In: 2013 Proceedings of the IEEE INFOCOM, pp. 1609–1617. IEEE (2013)Google Scholar
  3. 3.
    Chen, F., Kodialam, M., Lakshman, T.: Joint scheduling of processing and shuffle phases in MapReduce systems. In: Proceedings of the IEEE INFOCOM, pp. 1143–1151. IEEE (2012)Google Scholar
  4. 4.
    Wang, Y., Wang, W., Ma, C., Meng, D.: Zput: a speedy data uploading approach for the distributed file system. In: 2013 IEEE International Conference on Cluster Computing (CLUSTER), pp. 1–5. IEEE (2013)Google Scholar
  5. 5.
    White, T.: Hadoop: The Definitive Guide: The Definitive Guide. O’Reilly Media, Inc., Sebastopol (2009)Google Scholar
  6. 6.
    Chen, S., Schlosser, S.W.: Map-reduce meets wider varieties of applications. Intel Research Pittsburgh, Technical report IRP-TR-08-05 (2008)Google Scholar
  7. 7.
    Saravana Kumar, P., Athigopal, M., Vetrivel, S.: Extract transform and load strategy for unstructured data into data warehouse using map reduce algorithm and big data analytics. IJIRCCE 02, 7456–7462 (2014)CrossRefGoogle Scholar
  8. 8.
    Sivasangari, A., Ajitha, P., Indira, K.: Air pollution monitoring and prediction using multi view hybrid model. Int. J. Eng. Adv. Technol. (IJEAT) 8(2S), 1370–1372. ISSN: 2249–8958Google Scholar
  9. 9.
    Smys, S., Wee, H.-M., Joo, M.: Introduction to the Special Section on Inventive Systems and Smart Cities, pp. 32–33 (2018)CrossRefGoogle Scholar
  10. 10.
    Brumancia, E., Samuel, S.J., Gomathi, R.M., Dhas, Y.M.: An effective study on data fusion models in wireless sensor networks. 13(2) (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • P. Ajitha
    • 1
    Email author
  • A. Sivasangari
    • 1
  • G. Lakshmi Mounica
    • 2
  • Lakshmi Prathyusha
    • 2
  1. 1.Department of Information and TechnologySathyabama Institute of Science and TechnologyChennaiIndia
  2. 2.Computer Science and EngineeringSathyabama Institute of Science and TechnologyChennaiIndia

Personalised recommendations