Road surface type classification based on inertial sensors and machine learning

A comparison between classical and deep machine learning approaches for multi-contextual real-world scenarios

Computing (2021)Cite this article

Abstract

The demand for several sources of situational data from the traffic environment has intensified in recent years, through the development of applications in intelligent transport systems (ITS), such as autonomous vehicles and advanced driver assistance systems. Among these situational data, the road surface type classification is one of the most important and can be used throughout the ITS domain. However, in order to have a wide application, the development of a safe and reliable model is necessary. Therefore, in addition to the application of safe technology, the model developed must operate correctly in different vehicles, with different driving styles and in different environments in which vehicles can travel to. For this purpose, in this work we collect nine datasets with contextual variations using inertial sensors, represented by accelerometers and gyroscopes. These data were produced in three different vehicles, with three different drivers, in three different environments in which there are three different surface types, in addition to variations in conservation state and presence of obstacles and anomalies, such as speed bumps and potholes. After a pre-processing step, these data were used in 34 different computational models for road surface type classification, employing both Classical Machine Learning and Deep Learning techniques. Through several experiments, we analyze the learning and generalization capacity of each technique. The best model developed was a CNN-based deep neural network, which obtained validation accuracy of 93.17%, classifying surfaces between segments of dirt, cobblestone or asphalt roads.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1

Source the authors

Fig. 2

Source the authors

Fig. 3

Source the authors

Fig. 4

Source the authors

Fig. 5

Source the authors

Fig. 6

Source the authors

Fig. 7

Source the authors

Fig. 8

Source the authors

Fig. 9

Source the authors

Fig. 10

Source the authors

Fig. 11

Source the authors

Notes

  1. 1.

    1g = 9.8 \(\mathrm{m/s}^2\).

  2. 2.

    https://github.com/Intelligent-Vehicle-Perception/Intelligent-Vehicle-Perception-Based-on-Inertial-Sensing-and-Artificial-Intelligence and https://codigos.ufsc.br/lapix/intelligent-vehicle-perception-based-on-inertial-sensing.

References

  1. 1.

    Ahmad W, Kazmi BM, Ali H (2019) Human activity recognition using multi-head CNN followed by lstm. In: 2019 15th international conference on emerging technologies (ICET), pp 1–6

  2. 2.

    Al-Yafeai D, Darabseh T, Mourad AI (2019) Quarter vs. half car model energy harvesting systems. In: 2019 Advances in science and engineering technology international conferences (ASET), pp 1–5. https://doi.org/10.1109/ICASET.2019.8714433

  3. 3.

    Alemayoh TT, Hoon Lee J, Okamoto S (2019) Deep learning based real-time daily human activity recognition and its implementation in a smartphone. In: 2019 16th international conference on ubiquitous robots (UR), pp 179–182

  4. 4.

    Alqudah YA, Sababha BH (2016) A statistical approach to estimating driving events by a smartphone. In: 2016 international conference on computational science and computational intelligence (CSCI), pp 1021–1025. https://doi.org/10.1109/CSCI.2016.0195

  5. 5.

    Andria G, Attivissimo F, Nisio AD, Lanzolla A, Pellegrino A (2016) Development of an automotive data acquisition platform for analysis of driving behavior. Measurement 93:278–287. https://doi.org/10.1016/j.measurement.2016.07.035

    Article  Google Scholar 

  6. 6.

    Bello-Salau H, Aibinu A, Onumanyi A, Onwuka E, Dukiya J, Ohize H (2018) New road anomaly detection and characterization algorithm for autonomous vehicles. Appl Comput Inform. https://doi.org/10.1016/j.aci.2018.05.002

    Article  Google Scholar 

  7. 7.

    Bose B, Dutta J, Ghosh S, Pramanick P, Roy S (2018) Smartphone based system for real-time aggressive driving detection and marking rash driving-prone areas. In: Proceedings of the workshop program of the 19th international conference on distributed computing and networking, workshops ICDCN ’18, pp 27:1–27:6. ACM, New York, NY, USA. https://doi.org/10.1145/3170521.3170549

  8. 8.

    Braga N (2017) Manual de Mecatrônica. Editora NCB. https://books.google.com.br/books?id=Zr3iBgAAQBAJ

  9. 9.

    Brownlee J (2020) Imbalanced classification with python: better metrics, balance skewed classes, cost-sensitive learning. Machine Learning Mastery https://books.google.com.br/books?id=jaXJDwAAQBAJ

  10. 10.

    Brunauer R, Rehrl K (2016) Supporting road maintenance with in-vehicle data: Results from a field trial on road surface condition monitoring. In: 2016 IEEE 19th international conference on intelligent transportation systems (ITSC), pp 2236–2241. https://doi.org/10.1109/ITSC.2016.7795917

  11. 11.

    Chen Y, Xue Y (2015) A deep learning approach to human activity recognition based on single accelerometer. In: 2015 IEEE international conference on systems, man, and cybernetics, pp 1488–1492

  12. 12.

    Deep S, Zheng X (2019) Hybrid model featuring CNN and LSTM architecture for human activity recognition on smartphone sensor data. In: 2019 20th international conference on parallel and distributed computing, applications and technologies (PDCAT), pp 259–264

  13. 13.

    Dixon P, Schütte K, Vanwanseele B, Jacobs J, Dennerlein J, Schiffman J, Fournier PA, Hu B (2019) Machine learning algorithms can classify outdoor terrain types during running using accelerometry data. Gait and Posture 74:176–181. https://doi.org/10.1016/j.gaitpost.2019.09.005

    Article  Google Scholar 

  14. 14.

    Douangphachanh V, Oneyama H (2013) Estimation of road roughness condition from smartphones under realistic settings. In: 2013 13th international conference on ITS telecommunications (ITST), pp 433–439. https://doi.org/10.1109/ITST.2013.6685585

  15. 15.

    Fernández A, García S, Galar M, Prati RC, Krawczyk B, Herrera F (2018) Learning from imbalanced data sets. Springer, Berlin

    Google Scholar 

  16. 16.

    Foley D (2019) K-means clustering. https://towardsdatascience.com/k-means-clustering-8e1e64c1561c

  17. 17.

    Goodfellow I, Bengio Y, Courville A (2016) Deep learning. Adaptive computation and machine learning series. MIT Press, London. https://books.google.com.br/books?id=Np9SDQAAQBAJ

  18. 18.

    Groves P (2013) Principles of GNSS, inertial, and multisensor integrated navigation systems, 2nd edn. GNSS/GPS. Artech House. https://books.google.com.br/books?id=t94fAgAAQBAJ

  19. 19.

    Gueta LB, Sato A (2017) Classifying road surface conditions using vibration signals. In: 2017 Asia-Pacific signal and information processing association annual summit and conference (APSIPA ASC), pp 039–043. https://doi.org/10.1109/APSIPA.2017.8281999

  20. 20.

    He H, Ma Y (2013) Imbalanced learning: foundations, algorithms, and applications. Wiley, London

    Google Scholar 

  21. 21.

    Hou Y, Gupta A, Guan T, Hu S, Su L, Qiao C (2017) Vehsense: slippery road detection using smartphones. In: 2017 IEEE 85th vehicular technology conference (VTC Spring), pp 1–5. https://doi.org/10.1109/VTCSpring.2017.8108301

  22. 22.

    Jones MT (2017) Arquiteturas de aprendizado profundo: O surgimento da inteligência artificial. https://www.ibm.com/developerworks/br/library/cc-machine-learning-deep-learning-architectures/index.html

  23. 23.

    Khaleghian S, Taheri S (2017) Terrain classification using intelligent tire. J Terramech 71:15–24. https://doi.org/10.1016/j.jterra.2017.01.005

    Article  Google Scholar 

  24. 24.

    Khandelwal R (2018) K-nearest neighbors (KNN). https://medium.com/datadriveninvestor/k-nearest-neighbors-knn-7b4bd0128da7

  25. 25.

    Krawczyk B (2016) Learning from imbalanced data: open challenges and future directions. Progr Artif Intell 5(4):221–232

    Article  Google Scholar 

  26. 26.

    Kuhn M, Johnson K et al (2013) Applied predictive modeling, vol 26. Springer, Berlin

    Google Scholar 

  27. 27.

    Kumar GA, Kumar AS, Kumar AA, Maharajothi T (2017) Road quality management system using mobile sensors. In: 2017 international conference on innovations in information, embedded and communication systems (ICIIECS), pp 1–6. https://doi.org/10.1109/ICIIECS.2017.8276014

  28. 28.

    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  29. 29.

    Li F, Zhang H, Che H, Qiu X (2016) Dangerous driving behavior detection using smartphone sensors. In: 2016 IEEE 19th international conference on intelligent transportation systems (ITSC), pp 1902–1907. https://doi.org/10.1109/ITSC.2016.7795864

  30. 30.

    Lima LC, Amorim VJP, Pereira IM, Ribeiro FN, Oliveira RAR (2016) Using crowdsourcing techniques and mobile devices for asphaltic pavement quality recognition. In: 2016 VI Brazilian symposium on computing systems engineering (SBESC), pp 144–149. https://doi.org/10.1109/SBESC.2016.029

  31. 31.

    Gopi VP (2017) Vehicle vibration signal processing for road surface monitoring. IEEE Sens J 17(16):5192–5197. https://doi.org/10.1109/JSEN.2017.2719865

    Article  Google Scholar 

  32. 32.

    Machin M, Sanguesa JA, Garrido P, Martinez FJ (2018) On the use of artificial intelligence techniques in intelligent transportation systems. In: 2018 IEEE wireless communications and networking conference workshops (WCNCW), pp 332–337

  33. 33.

    Mathew TV (2014) Transportation systems engineering: Intrusive technologies. https://nptel.ac.in/courses/105101008/downloads/cete_09.pdf

  34. 34.

    Mathew TV (2014) Transportation systems engineering: Non-intrusive technologies. https://nptel.ac.in/courses/105101008/downloads/cete_10.pdf

  35. 35.

    Menegazzo J, Von Wangenheim A (2018) Vehicular perception and proprioception based on inertial sensing: a systematic review. Technical report, Federal University of Santa Catarina—Brazilian Institute for Digital Convergence. https://doi.org/10.13140/RG.2.2.20113.07525

  36. 36.

    Menegazzo J, von Wangenheim A (2020) Vehicular perception based on inertial sensing: a structured mapping of approaches and methods. SN Comput Sci. https://doi.org/10.1007/s42979-020-00275-z

    Article  Google Scholar 

  37. 37.

    Nalavde R, Surve A (2015) Driver assistant services using ubiquitous smartphone. In: 2015 international conference on communications and signal processing (ICCSP), pp 0430–0434. https://doi.org/10.1109/ICCSP.2015.7322924

  38. 38.

    Nisbet R, Elder J, Miner G (2009) Handbook of statistical analysis and data mining applications. Elsevier, Amsterdam

    Google Scholar 

  39. 39.

    Pholprasit T, Choochaiwattana W, Saiprasert C (2015) A comparison of driving behaviour prediction algorithm using multi-sensory data on a smartphone. In: 2015 IEEE/ACIs 16th international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (SNPD), pp 1–6 . https://doi.org/10.1109/SNPD.2015.7176249

  40. 40.

    Prapulla SB, Rao SN, Herur VA (2017) Road quality analysis and mapping for faster and safer travel. In: 2017 international conference on energy, communication, data analytics and soft computing (ICECDS), pp 2487–2490. https://doi.org/10.1109/ICECDS.2017.8389899

  41. 41.

    Rateke T, Justen KA, von Wangenheim A (2019) Road surface classification with images captured from low-cost camera-road traversing knowledge (RTK) dataset. Rev Inform Teórica e Apl 26(3):50–64

    Article  Google Scholar 

  42. 42.

    Savera A, Zia A, Edhi MS, Tauseen M, Shamsi JA (2016) Bumpster: A mobile cloud computing system for speed breakers and ditches. In: 2016 IEEE 41st conference on local computer networks workshops (LCN Workshops), pp 65–71. https://doi.org/10.1109/LCN.2016.030

  43. 43.

    Sebastian B, Ben-Tzvi P (2019) Support vector machine based real-time terrain estimation for tracked robots. Mechatronics 62:102260. https://doi.org/10.1016/j.mechatronics.2019.102260

    Article  Google Scholar 

  44. 44.

    Shrestha A, Won M (2018) Deepwalking: enabling smartphone-based walking speed estimation using deep learning. In: 2018 IEEE global communications conference (GLOBECOM), pp 1–6

  45. 45.

    Shubham J (2018) Support vector machines (SVM). https://medium.com/coinmonks/support-vector-machines-svm-b2b433419d73

  46. 46.

    Singh B, Gupta A (2015) Recent trends in intelligent transportation systems: a review. J Transp Lit 9:30–34

    Article  Google Scholar 

  47. 47.

    Singh G, Bansal D, Sofat S, Aggarwal N (2017) Smart patrolling: an efficient road surface monitoring using smartphone sensors and crowdsourcing. Pervas Mobile Comput 40:71–88. https://doi.org/10.1016/j.pmcj.2017.06.002

    Article  Google Scholar 

  48. 48.

    Souza VM (2018) Asphalt pavement classification using smartphone accelerometer and complexity invariant distance. Eng Appl Artif Intell 74:198–211. https://doi.org/10.1016/j.engappai.2018.06.003

    Article  Google Scholar 

  49. 49.

    Tolentino-Rabelo R, Muñoz DM (2016) Online terrain classification for mobile robots using FPGAS. In: 2016 IEEE 7th Latin American symposium on circuits systems (LASCAS), pp 231–234

  50. 50.

    Wang J, Chen Y, Hao S, Peng X, Hu L (2019) Deep learning for sensor-based activity recognition: a survey. Pattern Recogn Lett 119:3–11. https://doi.org/10.1016/j.patrec.2018.02.010

    Article  Google Scholar 

  51. 51.

    Wang S, Kodagoda S, Shi L, Dai X (2018) Two-stage road terrain identification approach for land vehicles using feature-based and Markov random field algorithm. IEEE Intell Syst 33(1):29–39

    Article  Google Scholar 

  52. 52.

    Wang S, Kodagoda S, Shi L, Wang H (2017) Road-terrain classification for land vehicles: employing an acceleration-based approach. IEEE Veh Technol Mag 12(3):34–41

    Article  Google Scholar 

  53. 53.

    Wickramarathne T, Garg V, Bauer P (2018) On the use of 3-d accelerometers for road quality assessment. In: 2018 IEEE 87th vehicular technology conference (VTC Spring), pp 1–5. https://doi.org/10.1109/VTCSpring.2018.8417779

  54. 54.

    Yang J, Cheng K, Chen J, Zhou B, Li Q (2018) Smartphones based online activity recognition for indoor localization using deep convolutional neural network. In: 2018 ubiquitous positioning, indoor navigation and location-based services (UPINLBS), pp 1–7

  55. 55.

    Zaccone G, Karim M, Menshawy A (2017) Deep learning with TensorFlow. Packt Publishing. https://books.google.com.br/books?id=C0IwDwAAQBAJ

  56. 56.

    Zebin T, Balaban E, Ozanyan KB, Casson AJ, Peek N (2019) Implementation of a batch normalized deep lstm recurrent network on a smartphone for human activity recognition. In: 2019 IEEE EMBS international conference on biomedical health informatics (BHI), pp 1–4

  57. 57.

    Zebin T, Sperrin M, Peek N, Casson AJ (2018) Human activity recognition from inertial sensor time-series using batch normalized deep lstm recurrent networks. In: 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 1–4

Download references

Author information

Affiliations

  1. Graduate Program in Computer Science (PPGCC), Image Processing and Computer Graphics Lab (LAPIX), Brazilian National Institute for Digital Convergence (INCoD), Department of Informatics and Statistics (INE), Federal University of Santa Catarina (UFSC), Florianópolis, Santa Catarina, Brazil

    Jeferson Menegazzo & Aldo von Wangenheim

Authors
  1. Jeferson Menegazzo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aldo von Wangenheim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeferson Menegazzo.

Ethics declarations

Conflict of interest

The authors declare that they have no confict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Menegazzo, J., von Wangenheim, A. Road surface type classification based on inertial sensors and machine learning. Computing (2021). https://doi.org/10.1007/s00607-021-00914-0

Download citation

Keywords

  • Road conditions
  • Road surface type
  • Intelligent transport systems
  • Inertial sensors
  • Pattern recognition
  • Machine learning

Mathematics Subject Classification

  • 68Q07
  • 68T05
  • 68T07
  • 68T10
  • 68T40