Abstract
Traffic information retrieval and data mining are not only the hotspots and key techniques in the intelligent transportation, but also the research issue of massive data’s distributed processing. With the development of urban traffic acquisition technology, the traffic data have increased to PB level. In order to manage these traffic data effectively and serve for intelligent transportation, we need to use efficient algorithm to process them in the distributed environment. In a distributed platform, this paper optimizes the Hadoop schedule algorithm that is used in processing traffic data and makes up the shortcomings of real-time traditional algorithms. The results of experiments show that the optimized scheduling algorithm used in a distributed environment, whether it is compute-intensive or I/O-intensive, has the most minimum calculation time, the best performance, better capacity of processing the traffic data, and better real time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, X., Lu, F., Zhang, H., et al.: Estimating Beijing’s travel delays at intersections with floating car data. In: Proceedings of the 5th ACM SIGSPATIAL International Workshop on Computational Transportation Science, ACM, pp. 14–19 (2012)
Xu, X., Wu, J., Yang, G.: Mass data processing system based on large scale low cost computing platform. Appl. Res. Comput. 29(2), 049 (2012)
Edwards, M., Rambani, A., Zhu, Y., et al.: Design of Hadoop-based framework for analytics of large synchrophasor datasets. Procedia. Comput. Sci. 12, 254–258 (2012)
Fischer, M.J., Su, X., Yin, Y.: Assigning tasks for efficiency in Hadoop. In: Proceedings of the 22nd ACM Symposium on Parallelism in Algorithms and Architectures, ACM, pp. 30–39 (2010)
Deng, C., Fan, T., Gao, F.: Resource scheduler algorithm based on statistical optimization under Hadoop. Appl. Res. Comput. 30(2), 417–419 (2013)
Wang, F.: Scheduling algorithm of Hadoop cluster jobs. Programmer 12, 1–19 (2009)
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
You, H.H., Yang, C.C., Huang, J.L.: A load-aware scheduler for MapReduce framework in heterogeneous cloud environments. In: Proceedings of the ACM Symposium on Applied Computing, ACM, pp. 127–132 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer India
About this paper
Cite this paper
Sun, W., Wang, X. (2014). The Optimization of Hadoop Scheduling Algorithms on Distributed System for Processing Traffic Information. In: Patnaik, S., Li, X. (eds) Proceedings of International Conference on Soft Computing Techniques and Engineering Application. Advances in Intelligent Systems and Computing, vol 250. Springer, New Delhi. https://doi.org/10.1007/978-81-322-1695-7_44
Download citation
DOI: https://doi.org/10.1007/978-81-322-1695-7_44
Published:
Publisher Name: Springer, New Delhi
Print ISBN: 978-81-322-1694-0
Online ISBN: 978-81-322-1695-7
eBook Packages: EngineeringEngineering (R0)