Advertisement

Autonomous driving: cognitive construction and situation understanding

  • Shitao Chen
  • Zhiqiang Jian
  • Yuhao Huang
  • Yu Chen
  • Zhuoli Zhou
  • Nanning ZhengEmail author
Review

Abstract

Autonomous vehicle is a kind of typical complex artificial intelligence system. In current research of autonomous driving, the most widely adopted technique is to use a basic framework of serial information processing and computations, which consists of four modules: perception, planning, decision-making, and control. However, this framework based on data-driven computing performs low computational efficiency, poor environmental understanding and self-learning ability. A neglected problem has long been how to understand and process environmental perception data from the sensors referring to the cognitive psychology level of the human driving process. The key to solving this problem is to construct a computing model with selective attention and self-learning ability for autonomous driving, which is supposed to possess the mechanism of memorizing, inferring and experiential updating, enabling it to cope with traffic scenarios with high noise, dynamic, and randomness. In addition, for the process of understanding traffic scenes, the efficiency of event-related mechanism is more significant than single-attribute scenario perception data. Therefore, an effective self-driving method should not be confined to the traditional computing framework of ‘perception, planning, decision-making, and control’. It is necessary to explore a basic computing framework that conforms to human driver’s attention, reasoning, learning, and decision-making mechanism with regard to traffic scenarios and build an autonomous system inspired by biological intelligence. In this article, we review the basic methods and main progress in current data-driven autonomous driving technologies, deeply analyze the limitations and major problems faced by related algorithms. Then, combined with authors’ research, this study discusses how to implement a basic cognitive computing framework of self-driving with selective attention and an event-driven mechanism from the basic viewpoint of cognitive science. It further describes how to use multi-sensor and graph data with semantic information (such as traffic maps and a spatial correlation of events) to realize the associative representations of objects and drivable areas, as well as the intuitive reasoning method applied to understanding the situations in different traffic scenarios. The computing framework of autonomous driving based on a selective attention mechanism and intuitive reasoning discussed in this study can adapt to a more complex, open, and dynamic traffic environment.

Keywords

autonomous driving event-driven mechanism cognitive construction situation understanding intuitive reasoning 

Notes

Acknowledgements

This work was partially supported by National Natural Science Foundation of China (Grant Nos. 61773312, 61790563).

References

  1. 1.
    Thrun S, Montemerlo M, Dahlkamp H, et al. Stanley: the robot that won the DARPA grand challenge. J Field Robot, 2006, 23: 661–692CrossRefGoogle Scholar
  2. 2.
    Miller G A. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev, 1956, 63: 81–97CrossRefGoogle Scholar
  3. 3.
    Kahneman D, Treisman A, Gibbs B J. The reviewing of object files: object-specific integration of information. Cogn Psychol, 1992, 24: 175–219CrossRefGoogle Scholar
  4. 4.
    Kahneman D, Frederick S. Representativeness revisited: attribute substitution in intuitive judgment. In: Heuristics and Biases: the Psychology of Intuitive Judgment. New York: Cambridge University Press, 2002. 49–81CrossRefGoogle Scholar
  5. 5.
    Jang Y, Song Y, Yu Y, et al. TGIF-QA: toward spatio-temporal reasoning in visual question answering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, 2017. 2758–2766Google Scholar
  6. 6.
    Ojala T, Pietikainen M, Harwood D. Performance evaluation of texture measures with classification based on kullback discrimination of distributions. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, Jerusalem, 1994. 582–585Google Scholar
  7. 7.
    Ojala T, Pietikainen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recogn, 1996, 29: 51–59CrossRefGoogle Scholar
  8. 8.
    Zhao L, Thorpe C E. Stereo- and neural network-based pedestrian detection. IEEE Trans Intell Transp Syst, 2000, 1: 148–154CrossRefGoogle Scholar
  9. 9.
    Yuan Y, Xiong Z T, Wang Q. An incremental framework for video-based traffic sign detection, tracking, and recognition. IEEE Trans Intell Transp Syst, 2017, 18: 1918–1929CrossRefGoogle Scholar
  10. 10.
    Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014. 580–587Google Scholar
  11. 11.
    Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, 2015. 1440–1448Google Scholar
  12. 12.
    Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, 2015. 91–99Google Scholar
  13. 13.
    Liu W, Anguelov D, Erhan D, et al. Ssd: single shot multibox detector. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2016. 21–37Google Scholar
  14. 14.
    Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 779–788Google Scholar
  15. 15.
    Redmon J, Farhadi A. Yolo9000: better, faster, stronger. 2017. ArXiv: 1612.08242Google Scholar
  16. 16.
    Wu B C, Iandola F, Jin P H, et al. Squeezedet: unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, 2017Google Scholar
  17. 17.
    Chen X Z, Ma H M, Wan J, et al. Multi-view 3D object detection network for autonomous driving. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 3Google Scholar
  18. 18.
    Ku J, Mozifian M, Lee J, et al. Joint 3D proposal generation and object detection from view aggregation. 2017. ArXiv: 1712.02294Google Scholar
  19. 19.
    Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? the kitti vision benchmark suite. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012. 3354–3361Google Scholar
  20. 20.
    Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. 3431–3440Google Scholar
  21. 21.
    Badrinarayanan V, Kendall A, Cipolla R. Segnet: a deep convolutional encoder-decoder architecture for image segmentation. 2015. ArXiv: 1511.00561Google Scholar
  22. 22.
    Chen L-C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected crfs. 2014. ArXiv: 1412.7062Google Scholar
  23. 23.
    Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell, 2018, 40: 834–848CrossRefGoogle Scholar
  24. 24.
    Wang Q, Gao J Y, Yuan Y. A joint convolutional neural networks and context transfer for street scenes labeling. IEEE Trans Intell Transp Syst, 2018, 19: 1457–1470CrossRefGoogle Scholar
  25. 25.
    Oliveira G L, Burgard W, Brox T. Efficient deep models for monocular road segmentation. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016. 4885–4891Google Scholar
  26. 26.
    Dai J F, He K M, Sun J. Instance-aware semantic segmentation via multi-task network cascades. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 3150–3158Google Scholar
  27. 27.
    He K M, Gkioxari G, Dollar P, et al. Mask r-cnn. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), 2017. 2980–2988Google Scholar
  28. 28.
    Li Y, Qi H Z, Dai J F, et al. Fully convolutional instance-aware semantic segmentation. 2016. ArXiv: 1611.07709Google Scholar
  29. 29.
    Bosse M, Zlot R. Continuous 3D scan-matching with a spinning 2D laser. In: Proceedings of IEEE International Conference on Robotics and Automation, 2009. 4312–4319Google Scholar
  30. 30.
    Baldwin I, Newman P. Laser-only road-vehicle localization with dual 2D push-broom lidars and 3D priors. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012. 2490–2497Google Scholar
  31. 31.
    Pfrunder A, Borges P V K, Romero A R, et al. Real-time autonomous ground vehicle navigation in heterogeneous environments using a 3D lidar. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017. 2601–2608Google Scholar
  32. 32.
    Liu Z Y, Yu S Y, Wang X, et al. Detecting drivable area for self-driving cars: an unsupervised approach. 2017. ArXiv: 1705.00451Google Scholar
  33. 33.
    Satzoda R K, Sathyanarayana S, Srikanthan T, et al. Hierarchical additive hough transform for lane detection. IEEE Embedded Syst Lett, 2010, 2: 23–26CrossRefGoogle Scholar
  34. 34.
    Huang Y H, Chen S T, Chen Y, et al. Spatial-temproal based lane detection using deep learning. In: Proceedings of IFIP International Conference on Artificial Intelligence Applications and Innovations, 2018. 143–154Google Scholar
  35. 35.
    Lee S, Kweon I S, Kim J, et al. Vpgnet: vanishing point guided network for lane and road marking detection and recognition. In: Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV), 2017. 1965–1973Google Scholar
  36. 36.
    Pan X G, Shi J P, Luo P, et al. Spatial as deep: spatial CNN for traffic scene understanding. 2017. ArXiv: 1712.06080Google Scholar
  37. 37.
    Zhang G, Zheng N N, Cui C, et al. An efficient road detection method in noisy urban environment. In: Proceedings of 2009 IEEE Intelligent Vehicles Symposium. New York: IEEE, 2009. 556–561CrossRefGoogle Scholar
  38. 38.
    Lv X, Liu Z Y, Xin J M, et al. A novel approach for detecting road based on two-stream fusion fully convolutional network. In: Intelligent Vehicles. New York: IEEE, 2018Google Scholar
  39. 39.
    Chen Z, Chen Z J. Rbnet: a deep neural network for unified road and road boundary detection. In: Proceedings of International Conference on Neural Information Processing. Berlin: Springer, 2017. 677–687CrossRefGoogle Scholar
  40. 40.
    Munoz-Bulnes J, Fernandez C, Parra I, et al. Deep fully convolutional networks with random data augmentation for enhanced generalization in road detection. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 366–371Google Scholar
  41. 41.
    Lv X, Liu Z Y, Xin J M, et al. A novel approach for detecting road based on two-stream fusion fully convolutional network. In: Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV). New York: IEEE, 2018. 1464–1469CrossRefGoogle Scholar
  42. 42.
    Warren C W. Fast path planning using modified A* method. In: Proceedings of IEEE International Conference on Robotics and Automation, 1993. 662–667Google Scholar
  43. 43.
    Zeng W, Church R L. Finding shortest paths on real road networks: the case for A*. Int J Geographical Inf Sci, 2009, 23: 531–543CrossRefGoogle Scholar
  44. 44.
    Sislak D, Volf P, Pěchouček M. Accelerated A* path planning. In: Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, Budapest, 2009. 1133–1134Google Scholar
  45. 45.
    LaValle S M. Rapidly-Exploring Random Trees: a New Tool for Path Planning. Technical Report (TR 98-11), Iowa State University, 1998Google Scholar
  46. 46.
    Kuffner J J, LaValle S M. Rrt-connect: an efficient approach to single-query path planning. In: Proceedings of IEEE International Conference on Robotics and Automation, 2000. 995–1001Google Scholar
  47. 47.
    Bohlin R, Kavraki L E. Path planning using lazy prm. In: Proceedings of IEEE International Conference on Robotics and Automation, 2000. 521–528Google Scholar
  48. 48.
    Barraquand J, Langlois B, Latombe J C. Numerical potential field techniques for robot path planning. IEEE Trans Syst Man Cybern, 1992, 22: 224–241MathSciNetCrossRefGoogle Scholar
  49. 49.
    Yang S X, Luo C. A neural network approach to complete coverage path planning. IEEE Trans Syst Man Cybern B, 2004, 34: 718–724CrossRefGoogle Scholar
  50. 50.
    Ferrer G, Sanfeliu A. Bayesian human motion intentionality prediction in urban environments. Pattern Recogn Lett, 2014, 44: 134–140CrossRefGoogle Scholar
  51. 51.
    Ghori O, Mackowiak R, Bautista M, et al. Learning to forecast pedestrian intention from pose dynamics. In: Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV), 2018Google Scholar
  52. 52.
    Ma W-C, Huang D-A, Lee N, et al. Forecasting interactive dynamics of pedestrians with fictitious play. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 4636–4644Google Scholar
  53. 53.
    Pfeiffer M, Schaeuble M, Nieto J, et al. From perception to decision: a data-driven approach to end-to-end motion planning for autonomous ground robots. In: Proceedings of 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017. 1527–1533Google Scholar
  54. 54.
    Kim B, Kang C M, Lee S H, et al. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network. 2017. ArXiv: 1704.07049Google Scholar
  55. 55.
    Takahashi A, Hongo T, Ninomiya Y, et al. Local path planning and motion control for agv in positioning. In: Proceedings of IEEE/RSJ International Workshop on Intelligent Robots and Systems, 1989. 392–397Google Scholar
  56. 56.
    Piazzi A, Bianco C G L. Quintic g/sup 2/-splines for trajectory planning of autonomous vehicles. In: Proceedings of the IEEE Intelligent Vehicles Symposium, 2000. 198–203Google Scholar
  57. 57.
    Komoriya K, Tanie K. Trajectory design and control of a wheel-type mobile robot using b-spline curve. In: Proceedings of IEEE/RSJ International Workshop on Intelligent Robots and Systems, 1989. 398–405Google Scholar
  58. 58.
    Holger B, Dennis N, Marius Z J, et al. From G2 to G3 continuity: continuous curvature rate steering functions for sampling-based nonholonomic motion planning. In: Proceedings of Intelligent Vehicles. New York: IEEE, 2018Google Scholar
  59. 59.
    Petereit J, Emter T, Frey C W, et al. Application of hybrid A* to an autonomous mobile robot for path planning in unstructured outdoor environments. In: Proceedings of the 7th German Conference on Robotics, 2012. 1–6Google Scholar
  60. 60.
    Veres S M, Molnar L, Lincoln N K, et al. Autonomous vehicle control systemsa review of decision making. In: Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 2011. 225: 155–195Google Scholar
  61. 61.
    Lee D, Yannakakis M. Principles and methods of testing finite state machines-a survey. Proc IEEE, 1996, 84: 1090–1123CrossRefGoogle Scholar
  62. 62.
    Montemerlo M, Becker J, Bhat S, et al. Junior: the stanford entry in the urban challenge. J Field Robot, 2008, 25: 569–597CrossRefGoogle Scholar
  63. 63.
    Feinberg E A, Shwartz A. Handbook of Markov Decision Processes: Methods and Applications. Berlin: Springer Science & Business Media, 2012zbMATHGoogle Scholar
  64. 64.
    Ulbrich S, Maurer M. Probabilistic online pomdp decision making for lane changes in fully automated driving. In: Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems (ITSC), 2013. 2063–2067Google Scholar
  65. 65.
    Brechtel S, Gindele T, Dillmann R. Probabilistic decision-making under uncertainty for autonomous driving using continuous pomdps. In: Proceedings of IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), 2014. 392–399Google Scholar
  66. 66.
    van Otterlo M, Wiering M. Reinforcement learning and markov decision processes. In: Proceedings of Reinforcement Learning. Belin: Springer, 2012. 3–42CrossRefGoogle Scholar
  67. 67.
    Morton J, Wheeler T A, Kochenderfer M J. Analysis of recurrent neural networks for probabilistic modeling of driver behavior. IEEE Trans Intell Transp Syst, 2017, 18: 1289–1298CrossRefGoogle Scholar
  68. 68.
    Xu L H, Wang Y Z, Sun H B, et al. Integrated longitudinal and lateral control for Kuafu-II autonomous vehicle. IEEE Trans Intell Transp Syst, 2016, 17: 2032–2041CrossRefGoogle Scholar
  69. 69.
    Coulter R C. Implementation of the Pure Pursuit Path Tracking Algorithm. Technical Report, Carnegie-Mellon UNIV Pittsburgh PA Robotics INST, 1992Google Scholar
  70. 70.
    Camacho E F, Alba C B. Model Predictive Control. Berlin: Springer Science & Business Media, 2013Google Scholar
  71. 71.
    Rasekhipour Y, Khajepour A, Chen S K, et al. A potential field-based model predictive path-planning controller for autonomous road vehicles. IEEE Trans Intell Transp Syst, 2017, 18: 1255–1267CrossRefGoogle Scholar
  72. 72.
    Varshney P K. Multisensor data fusion. Electron Commun Eng J, 1997, 9: 245–253CrossRefGoogle Scholar
  73. 73.
    Hall D L, Llinas J. An introduction to multisensor data fusion. Proc IEEE, 1997, 85: 6–23CrossRefGoogle Scholar
  74. 74.
    Zhang Q, Liu Y, Blum R S, et al. Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: a review. Inf Fusion, 2018, 40: 57–75CrossRefGoogle Scholar
  75. 75.
    Liu Y H, Fan X Q, Lv C, et al. An innovative information fusion method with adaptive Kalman filter for integrated INS/GPS navigation of autonomous vehicles. Mech Syst Signal Process, 2018, 100: 605–616CrossRefGoogle Scholar
  76. 76.
    Behrendt K, Novak L, Botros R. A deep learning approach to traffic lights: detection, tracking, and classification. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2017. 1370–1377Google Scholar
  77. 77.
    Haberjahn M, Kozempel K. Multi level fusion of competitive sensors for automotive environment perception. In: Proceedings of 16th International Conference on Information Fusion (FUSION), 2013. 397–403Google Scholar
  78. 78.
    Scheunert U, Lindner P, Richter E, et al. Early and multi level fusion for reliable automotive safety systems. In: Proceedings of Intelligent Vehicles Symposium. New York: IEEE, 2007. 196–201Google Scholar
  79. 79.
    Rodrfguez-Garavito C H, Ponz A, García F, et al. Automatic laser and camera extrinsic calibration for data fusion using road plane. In: Proceedings of the 17th International Conference on Information Fusion (FUSION), 2014. 1–6Google Scholar
  80. 80.
    Park Y, Yun S, Won C S, et al. Calibration between color camera and 3D LIDAR instruments with a polygonal planar board. Sensors, 2014, 14: 5333–5353CrossRefGoogle Scholar
  81. 81.
    Wang X, Xu L H, Sun H B, et al. On-road vehicle detection and tracking using MMW radar and monovision fusion. IEEE Trans Intell Transp Syst, 2016, 17: 2075–2084CrossRefGoogle Scholar
  82. 82.
    Wang T, Xin J M, Zheng N N. A method integrating human visual attention and consciousness of radar and vision fusion for autonomous vehicle navigation. In: Proceedings of IEEE 4th International Conference on Space Mission Challenges for Information Technology (SMC-IT), 2011. 192–197Google Scholar
  83. 83.
    Zhu Z, Liu J L. Unsupervised extrinsic parameters calibration for multi-beam lidars. In: Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering, Paris, 2013. 1110–1113Google Scholar
  84. 84.
    Jiang J J, Xue P X, Chen S T, et al. Line feature based extrinsic calibration of lidar and camera. In: Proceedings of 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), 2018. 1–6Google Scholar
  85. 85.
    Sun S L, Deng Z L. Multi-sensor optimal information fusion Kalman filter. Automatica, 2004, 40: 1017–1023MathSciNetCrossRefzbMATHGoogle Scholar
  86. 86.
    Sarkka S, Vehtari A, Lampinen J. Rao-blackwellized particle filter for multiple target tracking. Inf Fusion, 2007, 8: 2–15CrossRefGoogle Scholar
  87. 87.
    Yang G S, Lin Y, Bhattacharya P. A driver fatigue recognition model based on information fusion and dynamic Bayesian network. Inf Sci, 2010, 180: 1942–1954CrossRefGoogle Scholar
  88. 88.
    Li Y B, Chen J, Ye F, et al. The improvement of DS evidence theory and its application in IR/MMW target recognition. J Sens, 2016, 2016: 1–15Google Scholar
  89. 89.
    Wu H D, Siegel M, Stiefelhagen R, et al. Sensor fusion using dempster-shafer theory [for context-aware hci]. In: Proceedings of the 19th IEEE Instrumentation and Measurement Technology Conference, 2002. 7–12Google Scholar
  90. 90.
    Murphy R R. Dempster-Shafer theory for sensor fusion in autonomous mobile robots. IEEE Trans Robot Automat, 1998, 14: 197–206CrossRefGoogle Scholar
  91. 91.
    Subramanian V, Burks T F, Dixon W E. Sensor fusion using fuzzy logic enhanced Kalman filter for autonomous vehicle guidance in citrus groves. Trans ASABE, 2009, 52: 1411–1422CrossRefGoogle Scholar
  92. 92.
    Klein L A, Klein L A. Sensor and data fusion: a tool for information assessment and decision making. In: Proceedings of SPIE, 2004Google Scholar
  93. 93.
    Eslami S M A, Rezende D J, Besse F, et al. Neural scene representation and rendering. Science, 2018, 360: 1204–1210CrossRefGoogle Scholar
  94. 94.
    Chen S T, Shang J H, Zhang S Y, et al. Cognitive map-based model: toward a developmental framework for self-driving cars. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 1–8Google Scholar
  95. 95.
    Chen S T, Zhang S Y, Shang J H, et al. Brain-inspired cognitive model with attention for self-driving cars. IEEE Trans Cogn Dev Syst, 2019, 11: 13–25CrossRefGoogle Scholar
  96. 96.
    Li D Y, Gao H B. A hardware platform framework for an intelligent vehicle based on a driving brain. Engineering, 2018, 4: 464–470CrossRefGoogle Scholar
  97. 97.
    Chen L. The topological approach to perceptual organization. Visual Cognition, 2005, 12: 553–637CrossRefGoogle Scholar
  98. 98.
    Eslami S M A, Heess N, Weber T, et al. Attend, infer, repeat: fast scene understanding with generative models. In: Proceedings of the 30th Conference on Neural Information Processing Systems, Barcelona, 2016. 3225–3233Google Scholar
  99. 99.
    Bar-Shalom Y, Daum F, Huang J. The probabilistic data association filter. IEEE Control Syst, 2009, 29: 82–100CrossRefzbMATHGoogle Scholar
  100. 100.
    Svensson L, Svensson D, Guerriero M, et al. Set JPDA filter for multitarget tracking. IEEE Trans Signal Process, 2011, 59: 4677–4691MathSciNetCrossRefzbMATHGoogle Scholar
  101. 101.
    Blackman S S. Multiple hypothesis tracking for multiple target tracking. IEEE Aerosp Electron Syst Mag, 2004, 19: 5–18CrossRefGoogle Scholar
  102. 102.
    Kim C, Li F X, Ciptadi A, et al. Multiple hypothesis tracking revisited. In: Proceedings of the IEEE International Conference on Computer Vision, 2015. 4696–4704Google Scholar
  103. 103.
    Kuhn H W. The Hungarian method for the assignment problem. Naval Res Logistics, 1955, 2: 83–97MathSciNetCrossRefzbMATHGoogle Scholar
  104. 104.
    Cho H, Seo Y-W, Kumar B V K V, et al. A multi-sensor fusion system for moving object detection and tracking in urban driving environments. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2014. 1836–1843Google Scholar
  105. 105.
    Chavez-Garcia R O, Aycard O. Multiple sensor fusion and classification for moving object detection and tracking. IEEE Trans Intell Transp Syst, 2016, 17: 525–534CrossRefGoogle Scholar
  106. 106.
    Göhring D, Wang M, Schnürmacher M, et al. Radar/lidar sensor fusion for car-following on highways. In: Proceedings of the 5th International Conference on Automation, Robotics and Applications (ICARA), 2011. 407–412Google Scholar
  107. 107.
    Fayad F, Cherfaoui V. Object-level fusion and confidence management in a multi-sensor pedestrian tracking system. In: Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2008. 58–63Google Scholar
  108. 108.
    Kim D Y, Jeon M. Data fusion of radar and image measurements for multi-object tracking via Kalman filtering. Inf Sci, 2014, 278: 641–652MathSciNetCrossRefGoogle Scholar
  109. 109.
    Govaers F, Koch W. An exact solution to track-to-track-fusion at arbitrary communication rates. IEEE Trans Aerosp Electron Syst, 2012, 48: 2718–2729CrossRefGoogle Scholar
  110. 110.
    Zhang Z Y, Fidler S, Urtasun R. Instance-level segmentation for autonomous driving with deep densely connected mrfs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 669–677Google Scholar
  111. 111.
    Sun Y X, Liu M, Meng M Q H. Improving RGB-D SLAM in dynamic environments: a motion removal approach. Robot Auton Syst, 2017, 89: 110–122CrossRefGoogle Scholar
  112. 112.
    Sun Y X, Liu M, Meng M Q H. Motion removal for reliable RGB-D SLAM in dynamic environments. Robotics Autonomous Syst, 2018, 108: 115–128CrossRefGoogle Scholar
  113. 113.
    Caron F, Duflos E, Pomorski D, et al. GPS/IMU data fusion using multisensor Kalman filtering: introduction of contextual aspects. Inf Fusion, 2006, 7: 221–230CrossRefGoogle Scholar
  114. 114.
    Suhr J K, Jang J, Min D, et al. Sensor fusion-based low-cost vehicle localization system for complex urban environments. IEEE Trans Intell Transp Syst, 2017, 18: 1078–1086CrossRefGoogle Scholar
  115. 115.
    Wan G W, Yang X L, Cai R L, et al. Robust and precise vehicle localization based on multi-sensor fusion in diverse city scenes. 2017. ArXiv: 1711.05805Google Scholar
  116. 116.
    Tamar A, Wu Y, Thomas G, et al. Value iteration networks. In: Advances in Neural Information Processing Systems, 2016. 2154–2162Google Scholar
  117. 117.
    Katsuki F, Constantinidis C. Bottom-up and top-down attention: different processes and overlapping neural systems. Neuroscientist, 2014, 20: 509–521CrossRefGoogle Scholar
  118. 118.
    Miller E K. Neurobiology: straight from the top. Nature, 1999, 401: 650–651CrossRefGoogle Scholar
  119. 119.
    Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Machine Intell, 1998, 20: 1254–1259CrossRefGoogle Scholar
  120. 120.
    Kadir T, Brady M. Saliency, scale and image description. Int J Comput Vision, 2001, 45: 83–105CrossRefzbMATHGoogle Scholar
  121. 121.
    Ba J, Mnih V, Kavukcuoglu K. Multiple object recognition with visual attention. 2014. ArXiv: 1412.7755Google Scholar
  122. 122.
    Hu J, Shen L, Sun G. Squeeze-and-excitation networks. 2017 ArXiv: 1709.01507Google Scholar
  123. 123.
    Fu J L, Zheng H L, Mei T. Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017. 3Google Scholar
  124. 124.
    Kuo W J, Sjostrom T, Chen Y P, et al. Intuition and deliberation: two systems for strategizing in the brain. Science, 2009, 324: 519–522CrossRefGoogle Scholar
  125. 125.
    Zheng N N, Liu Z Y, Ren P J, et al. Hybrid-augmented intelligence: collaboration and cognition. Front Inf Technol Electron Eng, 2017, 18: 153–179CrossRefGoogle Scholar
  126. 126.
    Zhao D B, Hu Z H, Xia Z P, et al. Full-range adaptive cruise control based on supervised adaptive dynamic programming. Neurocomputing, 2014, 125: 57–67CrossRefGoogle Scholar
  127. 127.
    Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature, 2015, 518: 529–533CrossRefGoogle Scholar
  128. 128.
    Silver D, Huang A, Maddison C J, et al. Mastering the game of Go with deep neural networks and tree search. Nature, 2016, 529: 484–489CrossRefGoogle Scholar
  129. 129.
    Gupta S, Davidson J, Levine S, et al. Cognitive mapping and planning for visual navigation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 7272–7281Google Scholar
  130. 130.
    Rusu A A, Rabinowitz N C, Desjardins G, et al. Progressive neural networks. 2016. ArXiv: 1606.04671Google Scholar
  131. 131.
    Rusu A A, Vecerik M, Rothorl T, et al. Sim-to-real robot learning from pixels with progressive nets. 2016. ArXiv: 1610.04286Google Scholar

Copyright information

© Science China Press and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Shitao Chen
    • 1
    • 2
  • Zhiqiang Jian
    • 1
    • 2
  • Yuhao Huang
    • 1
    • 2
  • Yu Chen
    • 1
    • 2
  • Zhuoli Zhou
    • 1
    • 2
  • Nanning Zheng
    • 1
    • 2
    Email author
  1. 1.Institute of Artificial Intelligence and RoboticsXi’an Jiaotong UniversityXi’anChina
  2. 2.National Engineering Laboratory for Visual Information Processing and ApplicationsXi’anChina

Personalised recommendations