Advertisement

Neural Computing and Applications

, Volume 31, Issue 12, pp 8871–8886 | Cite as

A versatile hardware/software platform for personalized driver assistance based on online sequential extreme learning machines

  • Inés del CampoEmail author
  • Victoria Martínez
  • Javier Echanobe
  • Estibalitz Asua
  • Raúl Finker
  • Koldo Basterretxea
Original Article
  • 25 Downloads

Abstract

In the present scenario of technological breakthroughs in the automotive industry, machine learning is greatly contributing to the development of safer and more comfortable vehicles. In particular, personalization of the driving experience using machine learning is an innovative trend that comprises the development of both customized driver assistance systems and in-cabin comfort features. In this work, a versatile hardware/software platform for personalized driver assistance, using online sequential extreme learning machines (OS-ELM), is presented. The system, based on a programmable system-on-chip (SoC), is able to recognize the driver and personalize the behavior of the car. The platform provides high speed, small size, efficient power consumption, and true capability for real-time adaptation (i.e., on-chip self-learning). In addition, due to the plasticity and scalability of the OS-ELM algorithm and the programmable nature of the SoC, this solution is flexible enough to cope with the incremental changes that the new generation of vehicles are demanding. The implementation details of a system, suitable for current levels of driving automation, are provided.

Keywords

Driver assistance systems (DASs) Extreme learning machine Online learning Multi-objective optimization Field-programmable gate arrays (FPGA) System-on-chip (SoC) 

Notes

Acknowledgements

This work was supported in part by the Spanish Ministry of Economy and Competitiveness (MINECO) under Grant TEC2013-42286-R and by the Basque Country University UPV/EHU under Grant PPG17/20.

Compliance with ethical standards

Conflict of Interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    Bengler K, Dietmayer K, Farber B, Maurer M, Stiller C, Winner H (2014) Three decades of driver assistance systems: review and future perspectives. IEEE Intell Transp Syst Mag 6(4):6–22.  https://doi.org/10.1109/MITS.2014.2336271 CrossRefGoogle Scholar
  2. 2.
    Elbanhawi M, Simic M, Jazar R (2015) In the passenger seat: investigating ride comfort measures in autonomous cars. IEEE Intell Transp Syst Mag 7(3):4–17.  https://doi.org/10.1109/MITS.2015.2405571 CrossRefGoogle Scholar
  3. 3.
    Ford Motor Company: Ford and Intel Research Demonstrates the Future of In-Car Personalization and Mobile Interior Imaging Technology (2014). https://media.ford.com/content/fordmedia/fna/us/en/news/2014/06/25/ford-and-intel-research-demonstrates-the-future-of-in-car-person.html. Accessed 11 Jan 2019
  4. 4.
    Wu JD, Ye SH (2009) Driver identification based on voice signal using continuous wavelet transform and artificial neural network techniques. Expert Syst Appl 36(2):1061–1069.  https://doi.org/10.1016/j.eswa.2007.11.003 CrossRefGoogle Scholar
  5. 5.
    Riener A, Fersha A (2008) Supporting implicit human-to-vehicle interaction: driver identification from sitting postures. In: Proceedings of the ISVCS. Dublin, Ireland, pp 22–24Google Scholar
  6. 6.
    Qian H, Ou Y, Wu X, Meng X, Xu Y (2010) Support vector machine for behavior-based driver identification system. J Robot 2010:11.  https://doi.org/10.1155/2010/397865 CrossRefGoogle Scholar
  7. 7.
    Öztürk E, Erzin E (2012) Driver status identification from driving behavior signals. In: Hansen J, Boyraz P, Takeda K, Abut H (eds) Digital signal processing for in-vehicle systems and safety. Springer, Berlin, pp 31–55CrossRefGoogle Scholar
  8. 8.
    Martínez MV, del Campo I, Echanobe J, Basterretxea K (2015) Driving behavior signals and machine learning: A personalized driver assistance system. In: IEEE 18th international conference on intelligent transportation systems, pp 2933–2940.  https://doi.org/10.1109/ITSC.2015.470
  9. 9.
    Martínez MV, Echanobe J, del Campo I (2016) Driver identification and impostor detection based on driving behavior signals. In: IEEE 19th international conference on intelligent transportation systems (ITSC), pp 372–378.  https://doi.org/10.1109/ITSC.2016.7795582
  10. 10.
    Jafarnejad S, Castignani G, Engel T (2017) Towards a real-time driver identification mechanism based on driving sensing data. In: IEEE 20th international conference on intelligent transportation systems (ITSC).  https://doi.org/10.1109/ITSC.2017.8317716
  11. 11.
    Jafarnejad S, Castignani G, Engel T (2018) Revisiting gaussian mixture models for driver identification. In: IEEE international conference on vehicular electronics and safety (ICVES).  https://doi.org/10.1109/ICVES.2018.8519588
  12. 12.
    Moreira-Matias L, Farah H (2017) On developing a driver identification methodology using in-vehicle data recorders. IEEE Trans Intell Transp Syst 18(9):2387–2396.  https://doi.org/10.1109/TITS.2016.2639361 CrossRefGoogle Scholar
  13. 13.
    Butakov V, Ioannou P (2015) Driving autopilot with personalization feature for improved safety and comfort. In: IEEE 18th international conference on intelligent transportation systems, pp 387–393.  https://doi.org/10.1109/ITSC.2015.72
  14. 14.
    Wang J, Zhang L, Zhang D, Li K (2013) An adaptive longitudinal driving assistance system based on driver characteristics. IEEE Trans Intell Transp Syst 14(1):1–12.  https://doi.org/10.1109/TITS.2012.2205143 CrossRefGoogle Scholar
  15. 15.
    Butakov VA, Ioannou P (2015) Personalized driver/vehicle lane change models for adas. IEEE Trans Veh Technol 64(10):4422–4431.  https://doi.org/10.1109/TVT.2014.2369522 CrossRefGoogle Scholar
  16. 16.
    Lexus: Programming Memory Seats and Pairing Smart Key in Lexus (2016). http://www.whylexus.com/programming-memory-seats-and-pairing-smart-key-in-lexus. Accessed 11 Jan 2019
  17. 17.
    Buick: Vehicle Personalization (2017). http://www.buiclub.com/info-1733.html. Accessed 11 Jan 2019
  18. 18.
    Chen Y, Li L (eds) (2014) Advances in intelligent vehicles: intelligent systems series. Academic Press, CambridgeGoogle Scholar
  19. 19.
    Ohn-Bar E, Trivedi MM (2016) Looking at humans in the age of self-driving and highly automated vehicles. IEEE Trans Intell Veh 1(1):90–104.  https://doi.org/10.1109/TIV.2016.2571067 CrossRefGoogle Scholar
  20. 20.
    Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybern) 42(2):513–529.  https://doi.org/10.1109/TSMCB.2011.2168604 CrossRefGoogle Scholar
  21. 21.
    Ding S, Xu X, Nie R (2014) Extreme learning machine and its applications. Neur Comput Appl 25(3):549–556.  https://doi.org/10.1007/s00521-013-1522-8 CrossRefGoogle Scholar
  22. 22.
    Tang J, Deng C, Huang GB (2016) Extreme learning machine for multilayer perceptron. IEEE Trans Neur Netw Learn Syst 27(4):809–821.  https://doi.org/10.1109/TNNLS.2015.2424995 MathSciNetCrossRefGoogle Scholar
  23. 23.
    Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neur Netw 17(6):1411–1423.  https://doi.org/10.1109/TNN.2006.880583 CrossRefGoogle Scholar
  24. 24.
    Renesas Electronics Coorporation: Advanced Driver Assistance System (ADAS) (2017). https://www.renesas.com/en-us/solutions/automotive/adas.html. Accessed 11 Jan 2019
  25. 25.
    NVIDIA Coorporation: NVIDIA DRIVE PX Scalable Supercomputer for Autonomous Driving (2017). http://www.nvidia.com/object/drive-px.html. Accessed 11 Jan 2019
  26. 26.
  27. 27.
    Mobileye: The Evolution of EyeQ (2017). http://www.mobileye.com/our-technology/evolution-eyeq-chip/. Accessed 11 Jan 2019
  28. 28.
    Texas Instruments: TDAx ADAS SoCs (2017). https://www.ti.com/processors/automotive-processors/tdax-adas-socs/overview.html. Accessed 11 Jan 2019
  29. 29.
    Gage T, Morris J (2015) The coming revolution in vehicle technology and its big implications. Xcell J 92:38–45Google Scholar
  30. 30.
    Johnson AP, Liu JX, Millard AG, Karim S, Tyrrell AM, Harkin J, Timmis J, McDaid LJ, Halliday DM (2018) Homeostatic fault tolerance in spiking neural networks: a dynamic hardware perspective. IEEE Trans Circuits Syst-I: Reg Pap 65(2):687–699.  https://doi.org/10.1109/TCSI.2017.2726763 CrossRefGoogle Scholar
  31. 31.
    Liu JX, Harkin J, Maguire LP, McDaid LJ, Wade JJ (2018) SPANNER: a self-repairing spiking neural network hardware architecture. IEEE Trans Neur Netw Learn Syst 29(4):1287–1300.  https://doi.org/10.1109/TNNLS.2017.2673021 CrossRefGoogle Scholar
  32. 32.
    Feng G, Huang GB, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357.  https://doi.org/10.1109/TNN.2009.2024147 CrossRefGoogle Scholar
  33. 33.
    Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes. The art of scientific computing, 3rd edn. Cambridge University Press, CambridgezbMATHGoogle Scholar
  34. 34.
    Pryke A, Mostaghim S, Nazemi A (2007) Heatmap visualization of population based multi objective algorithms. Springer, Berlin, pp 361–375.  https://doi.org/10.1007/978-3-540-70928-2_29 CrossRefGoogle Scholar
  35. 35.
    Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans Evol Comput 6(2):182–197.  https://doi.org/10.1109/4235.996017 CrossRefGoogle Scholar
  36. 36.
    Xilinx Inc.: All Programmable SoC with Hardware and Software Programmability (2017). https://www.xilinx.com/products/silicon-devices/soc/zynq-7000.html. Accessed 11 Jan 2019
  37. 37.
    Crockett LH, Elliot RA, Enderwitz MA, Stewart RW (2015) The Zynq book. University of Strathclyde. http://www.zynqbook.com/. Accessed 11 Jan 2019
  38. 38.
    Intel Corporation: Intel User-Customizable SoC-FPGAs (2017). https://www.altera.com/en_US/pdfs/literature/br/br-soc-fpga.pdf. Accessed 11 Jan 2019
  39. 39.
  40. 40.
    Finker R, del Campo I, Echanobe J, Martínez V (2014) An intelligent embedded system for real-time adaptive extreme learning machine. In: IEEE Symposium on intelligent embedded systems (IES), pp 61–69.  https://doi.org/10.1109/INTELES.2014.7008987
  41. 41.
    Frances-Villora J, Rosado-Muñoz A, Martínez-Villena JM, Bataller-Mompean M, Guerrero JF, Wegrzyn M (2016) Hardware implementation of real-time extreme learning machine in fpga: analysis of precision, resource occupation and performance. Comput Electr Eng 51:139–156.  https://doi.org/10.1016/j.compeleceng.2016.02.007 CrossRefGoogle Scholar
  42. 42.
    Bataller-Mompeán M, Martínez-Villena JM, Rosado-Muñoz A, Frances-Villora JV, Guerrero-Martínez JF, Wegrzyn M, Adamski M (2016) Support tool for the combined software/hardware design of on-chip elm training for slff neural networks. IEEE Trans Ind Inf 12(3):1114–1123.  https://doi.org/10.1109/TII.2016.2554521 CrossRefGoogle Scholar
  43. 43.
    Yeam TC, Ismail N, Mashiko K, Matsuzaki T (2017) Fpga implementation of extreme learning machine system for classification. In: TENCON 2017—IEEE region 10 conference, pp 1868–1873.  https://doi.org/10.1109/TENCON.2017.8228163
  44. 44.
    Xilinx Inc.: 7 Series DSP48E1 Slice. User Guide, ug479 (v1.9) edn. (2016). https://www.xilinx.com/support/documentation/user_guides/ug479_7Series_DSP48E1.pdf
  45. 45.
    Bosque G, del Campo I, Echanobe J (2014) Fuzzy systems, neural networks and neuro-fuzzy systems: a vision on their hardware implementation and platforms over two decades. Eng Appl Artif Intell 32:283–331.  https://doi.org/10.1016/j.engappai.2014.02.008 CrossRefGoogle Scholar
  46. 46.
    Xilinx Inc.: Zynq UltraScale+ MPSoC. Product Tables and Product Selection Guide (2018). https://www.xilinx.com/support/documentation/selection-guides/zynq-ultrascale-plus-product-selection-guide.pdf
  47. 47.
    Xilinx Inc.: Hardware Zone (2017). https://www.xilinx.com/products/design-tools/hardware-zone.html. Accessed 11 Jan 2019
  48. 48.
    Abut H, Erdogan H, Ercil A, Çürüklü B, Koman HC, Tas F, Argunsah AÖ, Cosar S, Akan B, Karabalkan H, Cökelek E, Ficici R, Sezer V, Danis S, Karaca M, Abbak M, Uzunbaş MG, Eritmen K, Imamoğlu M, Kalaycıoglu C (2009) Real-world data collection with UYANIK chap 3. Springer, Berlin, pp 23–44Google Scholar
  49. 49.
    Qi G, Du Y, Wu J, Hounsell N, Jia Y (2016) What is the appropriate temporal distance range for driving style analysis? IEEE Trans Intell Transp Syst 17(5):1393–1403.  https://doi.org/10.1109/TITS.2015.2502985 CrossRefGoogle Scholar
  50. 50.
    Wang J, Li K, Lu XY (2014) Comparative analysis and modeling of driver behavior characteristics chap 6. Academic Press, Cambridge, pp 159–198Google Scholar
  51. 51.
    Ostermeier R, Rühl G (2015) Method for controlling a ventilation/air-conditioning system of a vehicle, and vehicle having such a ventilation/air-conditioning system. https://www.google.com/patents/WO2015007481A1. WO Patent App. PCT/EP2014/063,320. Accessed 11 Jan 2019
  52. 52.
    Dykstra R, Wayne R (2016) Vehicle climate control method. U.S. Patent, US9242531 B2Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Electricity and Electronics, Faculty of Sciences and TechnologyUniversity of the Basque Country (UPV/EHU)LeioaSpain
  2. 2.CELESTIA S.L.U.SantanderSpain
  3. 3.Department of Electronics Technology, EUITIUniversity of the Basque Country (UPV/EHU)BilbaoSpain

Personalised recommendations