Abstract
Massive traffic data is produced constantly every day, causing problems in data integration, massive storage, high performance processing when applying conventional data management approaches. We propose a cloud computing based system H-TDMS (Hadoop based Traffic Data Management System) to capture, manage and process the traffic big data. H-TDMS designs a configurable tool for data integration, a scalable data scheme for data storage, a secondary index for fast search query, a computing framework for data analysis, and a web-based user-interface with data visualization service for user interaction. Experiments on actual traffic data show that H-TDMS achieves considerable performance in traffic big data management.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Apache hadoop. http://hadoop.apache.org. Accessed 10 Apr 2016
Apache hbase. http://hbase.apache.org. Accessed 10 Apr 2016
Apache spark. http://spark.apache.org. Accessed 10 Apr 2016
Apache sqoop. http://sqoop.apache.org. Accessed 10 Apr 2016
Postgresql. https://www.postgresql.org. Accessed 10 Apr 2016
Adiba, M., Castrejon-Castillo, J.C., Oviedo, J.A.E., Vargas-Solar, G., Zechinelli-Martini, J.L.: Big data management challenges, approaches, tools and their limitations. In: Yu, S., Lin, X., Misic, J., Shen, X.S., (eds.) Networking for Big Data, pp. 43–56. Chapman and Hall/CRC, February 2016
Aji, A., Wang, F., Vo, H., Lee, R., Liu, Q., Zhang, X., Saltz, J.: Hadoop GIS: a high performance spatial data warehousing system over mapreduce. Proc. VLDB Endowment 6(11), 1009–1020 (2013)
Benitez, I., Blasco, C., Mocholi, A., Quijano, A.: A two-step process for clustering electric vehicle trajectories. In: IEEE International Electric Vehicle Conference (IEVC), pp. 1–8. IEEE (2014)
Eldawy, A., Mokbel, M.F.: A demonstration of spatialhadoop: an efficient mapreduce framework for spatial data. Proc. VLDB Endowment 6(12), 1230–1233 (2013)
Kemp, G., Vargas-Solar, G., Da Silva, C.F., Ghodous, P., Collet, C., Lopezamaya, P.: Towards cloud big data services for intelligent transport systems. In: ISPE International Conference on Concurrent Engineering, vol. 2, pp. 377. IOS Press (2015)
Lee, K., Ganti, R.K., Srivatsa, M., Liu, L.: Efficient spatial query processing for big data. In: ACM International Conference on Advances in Geographic Information Systems, pp. 469–472. ACM (2014)
Lv, Y., Duan, Y., Kang, W., Li, Z., Wang, F.Y.: Traffic flow prediction with big data: a deep learning approach. IEEE Trans. Intell. Transp. Syst. 16(2), 865–873 (2015)
Moriya, K., Matsushima, S., Yamanishi, K.: Traffic risk mining from heterogeneous road statistics. In: IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 1–10. IEEE (2015)
Shah, N.K.: Big data and cloud computing: pitfalls and advantages in data management. In: International Conference on Computing for Sustainable Global Development (INDIACom), pp. 643–648. IEEE (2015)
Van Le, H., Takasu, A.: A scalable spatio-temporal data storage for intelligent transportation systems based on hbase. In: IEEE International Conference on Intelligent Transportation Systems (ITSC), pp. 2733–2738. IEEE (2015)
Xiong, G., Zhu, F., Dong, X., Fan, H., Hu, B., Kong, Q., Kang, W., Teng, T.: A kind of novel its based on space-air-ground big-data. IEEE Intell. Transp. Syst. Mag. 8(1), 10–22 (2016)
Xu, X., Dou, W.: An assistant decision-supporting method for urban transportation planning over big traffic data. In: Zu, Q., Hu, B., Gu, N., Seng, S. (eds.) HCC 2014. LNCS, vol. 8944, pp. 251–264. Springer, Heidelberg (2015)
Yu, J., Jiang, F., Zhu, T.: Rtic-c: a big data system for massive traffic information mining. In: International Conference on Cloud Computing and Big Data (CloudCom-Asia), pp. 395–402. IEEE (2013)
Yue, X., Cao, L., Chen, Y., Xu, B.: Multi-view actionable patterns for managing traffic bottleneck. In: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Zheng, X., Chen, W., Wang, P., Shen, D., Chen, S., Wang, X., Zhang, Q., Yang, L.: Big data for social transportation. IEEE Trans. Intell. Transp. Syst. 17(3), 620–630 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media Singapore
About this paper
Cite this paper
Hua, X., Wang, J., Lei, L., Zhou, B., Zhang, X., Liu, P. (2016). H-TDMS: A System for Traffic Big Data Management. In: Wu, J., Li, L. (eds) Advanced Computer Architecture. ACA 2016. Communications in Computer and Information Science, vol 626. Springer, Singapore. https://doi.org/10.1007/978-981-10-2209-8_8
Download citation
DOI: https://doi.org/10.1007/978-981-10-2209-8_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-2208-1
Online ISBN: 978-981-10-2209-8
eBook Packages: Computer ScienceComputer Science (R0)
