Brain-Inspired Perception, Motion and Control

  • Wenfeng WangEmail author
  • Xiangyang Deng
  • Liang Ding
  • Limin Zhang
Part of the Research on Intelligent Manufacturing book series (REINMA)


In this chapter, a possible solution for the future real implementation of brain-inspired perception (vision, audition and tactile), motion (the optimal path planning) and control (robots’ behavior management) is further presented. Based on the results from Chaps.  2 5, a conceptual model is established to evaluate cognition efficiency of the vision–brain, taking danger recognition as an example. Based on the vision hypothesis, the underwater robots with a deep vision system—single-shot multibox detector (SSD)—can preliminarily link the robotic vision cognition module with the brain-inspired perception, motion and control. Such a deep vision system can also be utilized to further enhance the performance of planetary exploration wheeled mobile robot in Chap.  5 or other robots. Core functional modules for future rebuilding a real vision–brain, along with the major principles to implement a real brain cognition, are presented, which include memory, thinking, imagination, feeling, speaking and other aspects associated with visual perception. Realization of a vision–brain not only includes the fusion of sensors, but also includes the fusion of feature and knowledge. Deep robotic vision is strongly suggested to be introduced into the future advanced robotic control system. At the end of this chapter, the intelligence extremes of the vision–brain and the necessity for the avoidance of robots’ threatening to human are theoretically analyzed, and therefore, the necessity to set an up limit for the development of artificial intelligence is theoretically demonstrated.


  1. 1.
    A. Bekhouch, I. Bouchrika, N. Doghmane, Improving view random access via increasing hierarchical levels for multi-view video coding. IEEE Trans. Consum. Electron. 62(4), 437–445 (2017)Google Scholar
  2. 2.
    R. Bhatt, R. Datta, A two-tier strategy for priority based critical event surveillance with wireless multimedia sensors. Wireless Netw. 22(1), 1–18 (2016)Google Scholar
  3. 3.
    J. Rajeshwari, K. Karibasappa, Adaboost modular tensor locality preservative projection: face detection in video using Adaboost modular-based tensor locality preservative projections. IET Comput. Vision 10(7), 670–678 (2017)Google Scholar
  4. 4.
    Y. Zhang, Q.Z. Li, F.N. Zang, Ship detection for visual maritime surveillance from non-stationary platforms. Ocean Eng. 141(9), 53–63 (2017)Google Scholar
  5. 5.
    A. Abrardo, M. Martalò, G. Ferrari, Information fusion for efficient target detection in large-scale surveillance wireless sensor networks. Inform. Fusion 38(11), 55–64 (2017)Google Scholar
  6. 6.
    S. Murayama, M. Haseyama, A note on traffic flow measurement for traffic surveillance video: reduction of performance degradation in various environments. Infect. Dis. Clin. North Am. 23(2), 209–214 (2009)Google Scholar
  7. 7.
    A.E. Maadi, X. Maldague, Outdoor infrared video surveillance: a novel dynamic technique for the subtraction of a changing background of IR images. Infrared Phys. Technol. 49(3), 261–265 (2007)Google Scholar
  8. 8.
    K. Srinivasan, K. Porkumaran, G. Sainarayanan, Background subtraction techniques for human body segmentation in indoor video surveillance. J. Sci. Ind. Res. 73(5), 342–345 (2014)Google Scholar
  9. 9.
    H. Sun, T. Tan, Spatio-temporal segmentation for video surveillance. Electron. Lett. 37(1), 20–21 (2000)Google Scholar
  10. 10.
    D.M.A. Akber, H.M. Julius, O. Chae, Background independent moving object segmentation for video surveillance. Ieice Trans. Commun. 92(2), 585–598 (2009)Google Scholar
  11. 11.
    A.N. Taeki, M.H. Kim, Context-aware video surveillance system. J. Electr. Eng. Technol. 7(1), 115–123 (2012)Google Scholar
  12. 12.
    A. Milosavljević, A. Dimitrijević, D. Rančić, GIS-augmented video surveillance. Int. J. Geogr. Inf. Sci. 24(9), 1415–1433 (2010)Google Scholar
  13. 13.
    J.S. Kim, H.Y. Dong, Y.H. Joo, Fast and robust algorithm of tracking multiple moving objects for intelligent video surveillance systems. IEEE Trans. Consum. Electron. 57(3), 1165–1170 (2011)Google Scholar
  14. 14.
    Z. Zhang, M. Wang, X. Geng, Crowd counting in public video surveillance by label distribution learning. Neurocomputing 166(1), 151–163 (2015)Google Scholar
  15. 15.
    H. Yoon, Y. Jung, S. Lee, An image sequence transmission method in wireless video surveillance systems. Wireless Pers. Commun. 82(3), 1225–1238 (2015)Google Scholar
  16. 16.
    W.F. Wang, X. Chen, G.W. Zhang, J. Qian, W. Peng, B.Q. Wu, H.W. Zheng, Precision security: integrating video surveillance with surrounding environment changes. Complexity, Article ID 2959030 (2018)Google Scholar
  17. 17.
    K.A. Niranjil, C. Sureshkumar, Background subtraction in dynamic environment based on modified adaptive GMM with TTD for moving object detection. J. Electr. Eng. Technol. 10(1), 372–378 (2015)Google Scholar
  18. 18.
    Q. Yan, L. Li, Kernel sparse tracking with compressive sensing. IET Comput. Vision 8(4), 305–315 (2014)Google Scholar
  19. 19.
    T. Kryjak, M. Komorkiewicz, M. Gorgon, Real-time implementation of foreground object detection from a moving camera using the vibe algorithm. Comput. Sci. Inf. Syst. 11(4), 1617–1637 (2014)Google Scholar
  20. 20.
    J. Cao, S. Kwong, R. Wang, A noise-detection based adaboost algorithm for mislabeled data. Pattern Recogn. 45(12), 4451–4465 (2012)zbMATHGoogle Scholar
  21. 21.
    M. Kimura, M. Shibata, Environment recognition using optical flow in an autonomous mobile robot. Parkinsonism & Related Disorders 14(8), S63–S64 (2008)Google Scholar
  22. 22.
    A. Temko, C. Nadeu, Classification of acoustic events using svm-based clustering schemes. Pattern Recogn. 39(4), 682–694 (2006)zbMATHGoogle Scholar
  23. 23.
    S.M. Erfani, S. Rajasegarar, S. Karunasekera, C. Leckie, High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recogn. 58(10), 121–134 (2016)Google Scholar
  24. 24.
    K. Kang, X. Wang, Fully convolutional neural networks for crowd segmentation. Comput. Sci. 49(1), 25–30 (2014)Google Scholar
  25. 25.
    M. Xu, J. Lei, Y. Shen, Hierarchical tracking with deep learning. J. Comput. Inf. Syst. 10(15), 6331–6338 (2014)Google Scholar
  26. 26.
    J. Hu, J. Lu, Y.P. Tan, Deep metric learning for visual tracking. IEEE Trans. Circuits Syst. Video Technol. 26(11), 2056–2068 (2016)Google Scholar
  27. 27.
    J. Kuen, K.M. Lim, C.P. Lee, Self-taught learning of a deep invariant representation for visual tracking via temporal slowness principle. Pattern Recogn. 48(10), 2964–2982 (2015)Google Scholar
  28. 28.
    R. Steen, A portable digital video surveillance system to monitor prey deliveries at raptor nests. J. Raptor Res. 43(1), 69–74 (2017)Google Scholar
  29. 29.
    B.G. Kim, Fast coding unit (CU) determination algorithm for high-efficiency video coding (HEVC) in smart surveillance application. J. Supercomputing 73(3), 1063–1084 (2017)Google Scholar
  30. 30.
    L. Chen, D. Zhu, J. Tian, J. Liu, Dust particle detection in traffic surveillance video using motion singularity analysis. Digit. Signal Proc. 58(3), 127–133 (2016)Google Scholar
  31. 31.
    S.A.A. Shah, M. Bennamoun, F. Boussaid, Iterative deep learning for image set based face and object recognition. Neurocomputing 174(1), 866–874 (2016)Google Scholar
  32. 32.
    I. Lenz, H. Lee, A. Saxena, Deep learning for detecting robotic grasps. Int. J. Robot. Res. 34(4–5), 705–724 (2013)Google Scholar
  33. 33.
    B. Kamsu-Foguem, D. Noyes, Graph-based reasoning in collaborative knowledge management for industrial maintenance. Comput. Ind. 64(8), 998–1013 (2013)Google Scholar
  34. 34.
    A. Ess, K. Schindler, B. Leibe, L. Van Gool, Object detection and tracking for autonomous navigation in dynamic environments. Int. J. Robot. Res. 29(14), 1707–1725 (2010)Google Scholar
  35. 35.
    J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You Only Look Once: Unified, Real-Time Object Detection. IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 779–788Google Scholar
  36. 36.
    J.H. Ruan, X.P. Wang, F.T.S. Chan, Y. Shi, Optimizing the intermodal transportation of emergency medical supplies using balanced fuzzy clustering. Int. J. Prod. Res. 54(14), 4368–4386 (2016)Google Scholar
  37. 37.
    J.H. Ruan, Y. Shi, Monitoring and assessing fruit freshness in IOT-based e-commerce delivery using scenario analysis and interval number approaches. Inf. Sci. 373(12), 557–570 (2016)Google Scholar
  38. 38.
    Z.H. Lv, S.U. Rehman, M.S.L. Khan, H. Li, Anaglyph 3D Stereoscopic Visualization of 2D Video Based on Fundamental Matrix. International Conference on Virtual Reality and Visualization (2013) pp. 305–308Google Scholar
  39. 39.
    J.H. Ruan, X.P. Wang, Y. Shi, A two-stage approach for medical supplies intermodal transportation in large-scale disaster responses. Int. J. Environ. Res. Publ. Health 11(11), 11081–11109 (2014)Google Scholar
  40. 40.
    H. Jiang, J.H. Ruan, Fuzzy evaluation on network security based on the new algorithm of membership degree transformation—m(1,2,3). J. Netw. 4(5), 324–331 (2009)Google Scholar
  41. 41.
    J.H. Ruan, P. Shi, C.C. Lim, X.P. Wang, Relief supplies allocation and optimization by interval and fuzzy number approaches. Inf. Sci. 303(3), 15–32 (2015)MathSciNetzbMATHGoogle Scholar
  42. 42.
    W. Otjacques, F.D. Laender, P. Kestemont, Discerning the causes of a decline in a common european fish, the roach (rutilus rutilus, l.): a modelling approach. Ecol. Model. 322(2), 92–100 (2016)Google Scholar
  43. 43.
    C.J. Littles, S.S. Pilyugin, T.K. Frazer, A combined inverse method and multivariate approach for exploring population trends of florida manatees. Marine Mammal Sci. 32(1), 122–140 (2016)Google Scholar
  44. 44.
    S. Santoro, A.J. Green, J. Figuerola, Immigration enhances fast growth of a newly established source population. Ecology 97(4), 1048–1057 (2016)Google Scholar
  45. 45.
    J.D. Smith, C. Zhang, Generalized lotka stability. Theor. Popul. Biol. 103(8), 38–43 (2015)zbMATHGoogle Scholar
  46. 46.
    R. Velik, A brain-inspired multimodal data mining approach for human activity recognition in elderly homes. J. Ambient Intell. Smart Environ. 6(4), 447–468 (2014)Google Scholar
  47. 47.
    J.J. Wong, S.Y. Cho, A brain-inspired framework for emotion recognition. Magn. Reson. Imaging 32(9), 1139–1155 (2006)Google Scholar
  48. 48.
    N. Ovcharova, F. Gauterin, Assessment of an adaptive predictive collision warning system based on driver’s attention detection. Clin. Exp. Metas. 8(2), 215–224 (2012)Google Scholar
  49. 49.
    A. Finn, K. Rogers, Accuracy requirements for unmanned aerial vehicle-based acoustic atmospheric tomography. J. Acoust. Soc. Am. 139(4), 2097–2097 (2016)Google Scholar
  50. 50.
    S. Kim, H. Oh, A. Tsourdos, Nonlinear model predictive coordinated standoff tracking of a moving ground vehicle. J. Guid. Control Dyn. 36(2), 557–566 (2013)Google Scholar
  51. 51.
    Z. Zheng, Y. Liu, X. Zhang, The more obstacle information sharing, the more effective real-time path planning? Knowl. Based Syst. 114(12), 36–46 (2016)Google Scholar
  52. 52.
    M.W. Whalen, D. Cofer, A. Gacek, Requirements and architectures for secure vehicles. IEEE Softw. 33(4), 22–25 (2016)Google Scholar
  53. 53.
    R. Czyba, G. Szafrański, A. Ryś, Design and control of a single tilt tri-rotor aerial vehicle. J. Intell. Rob. Syst. 84(1-4), 53–66 (2016)Google Scholar
  54. 54.
    X. Zhang, H. Duan, An improved constrained differential evolution algorithm for unmanned aerial vehicle global route planning. Appl. Soft Comput. 26(3), 270–284 (2015)Google Scholar
  55. 55.
    T. Uppal, S. Raha, S. Srivastava, Trajectory feasibility evaluation using path prescribed control of unmanned aerial vehicle in differential algebraic equations framework. Aeronaut. J. New Ser. 121(1240), 1–20 (2017)Google Scholar
  56. 56.
    A.V. Savkin, W. Chao, A framework for safe assisted navigation of semi-autonomous vehicles among moving and steady obstacles. Robotica 35(5), 981–1005 (2016)Google Scholar
  57. 57.
    Y.T. Tan, M. Chitre, F.S. Hover, Cooperative bathymetry-based localization using low-cost autonomous underwater vehicles. Auton. Robots 40(7), 1–19 (2015)Google Scholar
  58. 58.
    J.L. Crespo, A. Faiña, R.J. Duro, An adaptive detection/attention mechanism for real time robot operation. Neurocomputing 72(4–6), 850–860 (2009)Google Scholar
  59. 59.
    W. Barbara, Computational intelligence: from natural to artificial systems. Connection Sci. 14(2), 163–164 (2002)Google Scholar
  60. 60.
    E. Bonabeau, C. Meyer, Computational intelligence. A whole new way to think about business. Harvard Bus. Rev. 79(5), 106–114 (2001)Google Scholar
  61. 61.
    Y. Wang, D. Shen, E.K. Teoh, Lane detection using spline model. Pattern Recogn. Lett. 21(8), 677–689 (2000)Google Scholar
  62. 62.
    Z.W. Kim, Robust lane detection and tracking in challenging scenarios. IEEE Trans. Intell. Transp. Syst. 9(1), 16–26 (2008)Google Scholar
  63. 63.
    Q. Li, N. Zheng N, H. Cheng, Springrobot: a prototype autonomous vehicle and its algorithms for lane detection. IEEE Trans. Intell. Transp. Syst. 5(4), 300–308 (2004)Google Scholar
  64. 64.
    M. Dorigo, M. Birattari, C. Blum, Ant Colony Optimization and Computational Intelligence, vol. 49, no. 8 (Springer, Berlin, 1995), pp. 767–771Google Scholar
  65. 65.
    S. Garnier, J. Gautrais, G. Theraulaz, The biological principles of computational intelligence. Comput. Intell. 1(1), 3–31 (2007)Google Scholar
  66. 66.
    H.P. Liu, D. Guo, F.C. Sun, Object recognition using tactile measurements: kernel sparse coding methods. IEEE Trans. Instrum. Meas. 65(3), 656–665 (2016)Google Scholar
  67. 67.
    H.P. Liu, Y.L. Yu, F.C. Sun, J. Gu, Visual-tactile fusion for object recognition. IEEE Trans. Autom. Sci. Eng., 14(99), 1–13 (2017) Google Scholar

Copyright information

© Huazhong University of Science and Technology Press, Wuhan and Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  • Wenfeng Wang
    • 1
    Email author
  • Xiangyang Deng
    • 2
  • Liang Ding
    • 3
  • Limin Zhang
    • 2
  1. 1.CNITECH, Chinese Academy of SciencesInstitute of Advanced Manufacturing TechnologyNingboChina
  2. 2.Naval Aeronautical UniversityYantaiChina
  3. 3.Harbin Institute of TechnologyHarbinChina

Personalised recommendations