Hadoop Deep Neural Network for offending drivers

Abstract

Deep learning is recently regarded as the closest artificial intelligence model to human brain. It is about learning multiple levels of representation and abstraction that help to make sense of data such as images, sound, and text. Based on MapReduce framework and Hadoop distributed file system, this paper proposes a distributed approach for detect offending drivers and training the Deep Neural Network models such as Convolutional Neural Network (CNN) and Long Short Term Memory network (LSTM). Its implementation and performance are evaluated on Big Data platform Hadoop. The intelligence growing process of human brain requires learning from Big Data. The main contribution of this paper is that it is implemented to analyze traffic big data and to detect offending drivers in Hadoop by CNN with Support Vector Machine (SVM) and LSTM. The efficiency of the proposed method is computed by using experimental and theoretical analysis.

This is a preview of subscription content, access via your institution.

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

References

  1. Aghdam HH, Heravi EJ (2017) Guide to convolutional neural networks. Springer 10:973–978

    Google Scholar 

  2. Aoyama K (1997) Next Generation Universal Traffic Management System (UTMS'21) in Japan. Intelligent Transportation System, 1997. ITSC'97, IEEE Conference on IEEE.

  3. Arcos-García Á, Soilán M, Álvarez-García JA, Riveiro B (2017) Exploiting synergies of mobile mapping sensors and deep learning for traffic sign recognition systems. Expert Syst Appl 89:286–295

    Article  Google Scholar 

  4. Arcos-García Á, Álvarez-García JA, Soria-Morillo LM (2018a) Deep neural network for traffic sign recognition systems: An analysis of spatial transformers and stochastic optimisation methods. Neural Netw 99:158–165

    Article  Google Scholar 

  5. Arcos-García Á, Álvarez-García JA, Soria-Morillo LM (2018b) Evaluation of Deep Neural Networks for traffic sign detection systems. Neurocomputing 316:332–344

    Article  Google Scholar 

  6. Asadianfam S, Shamsi M, Kenari AR (2020a) TVD-MRDL: traffic violation detection system using MapReduce-based deep learning for large-scale data. Multimed Tools Appl 80:1–28

    Google Scholar 

  7. Asadianfam S, Shamsi M, Rasouli Kenari A (2020b) Big data platform of traffic violation detection system: identifying the risky behaviors of vehicle drivers. Multimed Tools Appl 29:24645–24684

    Article  Google Scholar 

  8. Atlas LE, Homma T, Marks RJ (1988) An artificial neural network for spatio-temporal bipolar patterns: application to phoneme classification. In: Anderson DZ (ed) Neural Information Processing Systems, pp 31–40

  9. Cantabella M, Martínez-España R, Ayuso B, Yáñez JA, Muñoz A (2019) Analysis of student behavior in learning management systems through a Big Data framework. Future Gener Comput Syst 90:262–272

    Article  Google Scholar 

  10. Chen M, Mao S, Liu Y (2014a) Big data: a survey. Mobile Netw Appl 19(2):171–209

    Article  Google Scholar 

  11. Chen M, Mao S, Zhang Y, Leung VC (2014b) Big data: related technologies, challenges and future prospects. Springer, Berlin

    Google Scholar 

  12. De Mauro A, Greco M, Grimaldi M, Ritala P (2018) Human resources for Big Data professions: a systematic classification of job roles and required skill sets. Inf Process Manag 54(5):807–817

    Article  Google Scholar 

  13. Dean J, Ghemawat S (2008) MapReduce: simplified data processing on large clusters. Commun ACM 51(1):107–113

    Article  Google Scholar 

  14. Dvornek NC, Ventola P, Pelphrey KA, Duncan JS (2017) Identifying autism from resting-state fMRI using long short-term memory networks. International workshop on machine learning in medical imaging. Springer, Berlin

    Google Scholar 

  15. Fukushima K, Miyake S (1982) Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. Competition and cooperation in neural nets. Springer, Berlin, pp 267–285

    Google Scholar 

  16. Furey TS, Cristianini N, Duffy N, Bednarski DW, Schummer M, Haussler D (2000) Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics 16(10):906–914

    Article  Google Scholar 

  17. Gantz J, Reinsel D (2011) Extracting value from chaos. IDC Iview 1142(2011):1–12

    Google Scholar 

  18. Goh YM, Chua D (2009) Case-based reasoning for construction hazard identification: case representation and retrieval. J Constr Eng Manag 135(11):1181–1189

    Article  Google Scholar 

  19. Goodfellow I, Bengio Y, Courville A (2016) Deep learning (adaptive computation and machine learning series). Adaptive Computation and Machine Learning series, MIT Press

  20. Guo S, Ding L, Luo H, Jiang X (2016) A Big-Data-based platform of workers’ behavior: observations from the field. Accid Anal Prev 93:299–309

    Article  Google Scholar 

  21. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  22. Hubel DH, Wiesel TN (1968) Receptive fields and functional architecture of monkey striate cortex. J Physiol 195(1):215–243

    Article  Google Scholar 

  23. Jeon S, Hong B (2016) Monte Carlo simulation-based traffic speed forecasting using historical big data. Future Gener Comput Syst 65:182–195

    Article  Google Scholar 

  24. Kim H (2019) Multiple vehicle tracking and classification system with a convolutional neural network. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-019-01429-5

    Article  Google Scholar 

  25. Kumar CR, Anuradha R (2020) Feature selection and classification methods for vehicle tracking and detection. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-020-01824-3

  26. Kutlu H, Avcı E (2019) A novel method for classifying liver and brain tumors using convolutional neural networks, discrete wavelet transform and long short-term memory networks. Sensors 19(9):1992

    Article  Google Scholar 

  27. Laney D (2001) 3D data management: controlling data volume, velocity and variety. META Group Res Note 6(70):1

    Google Scholar 

  28. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  29. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  30. Lotfi E (2012) Trajectory clustering and behaviour retrieval from traffic surveillance videos. Majlesi J Multimed Process 1:1–8

  31. Lu W (2019) Big data analytics to identify illegal construction waste dumping: A Hong Kong study. Resour Conserv Recycl 141:264–272

    Article  Google Scholar 

  32. McLauchlan P, Beymer D, Coifman B, Mali J (1997) A real-time computer vision system for measuring traffic parameters. cvpr, IEEE.

  33. Moghaddam AM, Ayati E (2014) Introducing a risk estimation index for drivers: a case of Iran. Saf Sci 62:90–97

    Article  Google Scholar 

  34. Munoz-Organero M, Ruiz-Blaquez R, Sánchez-Fernández L (2018) Automatic detection of traffic lights, street crossings and urban roundabouts combining outlier detection and deep learning classification techniques based on GPS traces while driving. Comput Environ Urban Syst 68:1–8

    Article  Google Scholar 

  35. Osman AMS (2018) A novel big data analytics framework for smart cities. Future Gener Comput Syst 91:620–633

    Article  Google Scholar 

  36. Park SH, Jung K, Hea JK, Kim HJ (1999) Vision-based traffic surveillance system on the internet. Computational Intelligence and Multimedia Applications, 1999. ICCIMA'99. Proceedings. Third International Conference on, IEEE.

  37. Patterson J, Gibson A (2017) Deep learning: a practitioner’s approach. O’Reilly Media Inc., Newton

    Google Scholar 

  38. Rahemi Z, Ajorpaz NM, Esfahani MS, Aghajani M (2017) Sensation-seeking and factors related to dangerous driving behaviors among Iranian drivers. Personality Individ Differ 116:314–318

    Article  Google Scholar 

  39. Rakotonirainy A, Schroeter R, Soro A (2014) Three social car visions to improve driver behaviour. Pervasive Mobile Comput 14:147–160

    Article  Google Scholar 

  40. Rao X, Lin F, Chen Z, Zhao J (2019) Distracted driving recognition method based on deep convolutional neural network. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-019-01597-4

  41. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117

    Article  Google Scholar 

  42. Secundo G, Del Vecchio P, Dumay J, Passiante G (2017) Intellectual capital in the age of Big Data: establishing a research agenda. J Intellect Cap 18(2):242–261

    Article  Google Scholar 

  43. Shvachko K, Kuang H, Radia S, Chansler R (2010). The hadoop distributed file system. Mass storage systems and technologies (MSST), 2010 IEEE 26th symposium on, IEEE.

  44. Tao D, Zhang R, Qu X (2017) The role of personality traits and driving experience in self-reported risky driving behaviors and accident risk among Chinese drivers. Accid Anal Prev 99:228–235

    Article  Google Scholar 

  45. Wallach I, Dzamba M, Heifets A (2015) AtomNet: a deep convolutional neural network for bioactivity prediction in structure-based drug discovery. arXiv preprint http://arxiv.org/abs/1510.02855

  46. White T (2012) Hadoop: the definitive guide. O’Reilly Media Inc, Newton

    Google Scholar 

  47. Yin L, Cheng Q, Wang Z, Shao Z (2015) ‘Big data’for pedestrian volume: Exploring the use of Google Street View images for pedestrian counts. Appl Geogr 63:337–345

    Article  Google Scholar 

  48. Yu Y, Li J, Wen C, Guan H, Luo H, Wang C (2016) Bag-of-visual-phrases and hierarchical deep models for traffic sign detection and recognition in mobile laser scanning data. ISPRS J Photogramm Remote Sens 113:106–123

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mahboubeh Shamsi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Asadianfam, S., Shamsi, M. & Rasouli Kenari, A. Hadoop Deep Neural Network for offending drivers. J Ambient Intell Human Comput (2021). https://doi.org/10.1007/s12652-021-02924-4

Download citation

Keywords

  • Deep learning
  • Object detection
  • Deep Neural Network
  • Convolutional Neural Network
  • Hadoop
  • Long Short Term Memory network