3D Vision-Based Autonomous Navigation System Using ANN and Kinect Sensor

  • Daniel Sales
  • Diogo Correa
  • Fernando S. Osório
  • Denis F. Wolf
Part of the Communications in Computer and Information Science book series (CCIS, volume 311)


In this paper, we present an autonomous navigation system based on a finite state machine (FSM) learned by an artificial neural network (ANN) in an indoor navigation task. This system uses a kinect as the only sensor. In the first step, the ANN is trained to recognize the different specific environment configurations, identifying the different environment situations (states) based on the kinect detections. Then, a specific sequence of states and actions is generated for any route defined by the user, configuring a path in a topological like map. So, the robot becomes able to autonomously navigate through this environment, reaching the destination after going through a sequence of specific environment places, each place being identified by its local properties, as for example, straight path, path turning to left, path turning to right, bifurcations and path intersections. The experiments were performed with a Pioneer P3-AT robot equipped with a kinect sensor in order to validate and evaluate this approach. The proposed method demonstrated to be a promising approach to autonomous mobile robots navigation.


Mobile Robotics Autonomous Navigation Kinect Artificial Neural Networks Finite State Machine 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zingg, S., Scaramuzza, D., Weiss, S., Siegwart, R.: MAV Navigation through Indoor Corridors Using Optical Flow. In: IEEE International Conference on Robotics and Automation (ICRA 2010), Anchorage, Alaska (May 2010)Google Scholar
  2. 2.
    Scaramuzza, D., Siegwart, R.: Appearance Guided Monocular Omnidirectional Visual Odometry for Outdoor Ground Vehicles. IEEE Transactions on Robotics 24(5) (October 2008)Google Scholar
  3. 3.
    Shinzato, P.Y., Wolf, D.F.: Features Image Analysis for Road Following Algorithm Using Neural Networks (September 2010)Google Scholar
  4. 4.
    Sales, D., Osório, F., Wolf, D.: Topological Autonomous Navigation for Mobile Robots in Indoor Environments using ANN and FSM. In: Proceedings of the I Brazilian Conference on Critical Embedded Systems (CBSEC), São Carlos, Brazil (2011)Google Scholar
  5. 5.
    Hopcroft, J.E., Ullman, J.D.: Introduction to Automata Theory, Languages and Computation. Addison - Wesley (1979)Google Scholar
  6. 6.
    Marino, A., Parker, L., Antonelli, G., Caccavale, F.: Behavioral Control for Multi-Robot Perimeter Patrol: A Finite State Automata approach. In: ICRA (2009)Google Scholar
  7. 7.
    Thrun, S., et al.: Stanley: The Robot that Won the DARPA Grand Challenge. Journal of Field Robotics 23(9), 661–692 (2006)CrossRefGoogle Scholar
  8. 8.
    Urmson, C., et al.: Autonomous driving in urban environments: Boss and the Urban Challenge. Journal of Field Robotics 25(8), 425–466 (August 2008); Special Issue on the 2007 DARPA Urban Challenge, Part IGoogle Scholar
  9. 9.
    Buehler, M., Iagnemma, K., Singh, S. (eds.): The 2005 DARPA Grand Challenge: The Great Robot Race (Springer Tracts in Advanced Robotics), 1st edn. Springer (October 2007)Google Scholar
  10. 10.
    Nefian, A.V., Bradski, G.R.: Detection of Drivable Corridors for Off-Road Autonomous Navigation. In: ICIP 2006: Proceedings of the IEEE International Conference on Image Processing, pp. 3025–3028 (2006)Google Scholar
  11. 11.
    Álvarez, J.M., López, A.M., Baldrich, R.: Illuminant Invariant Model-Based Road Segmentation. In: IEEE Intelligent Vehicles Symposium, Eindhoven, Netherlands (June 2008),
  12. 12.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
  13. 13.
    Bishop, R.: Intelligent vehicle applications worldwide. IEEE Intelligent Systems and their Applications 15(1), 78–81 (2000)CrossRefGoogle Scholar
  14. 14.
    Bishop, R.: A Survey of Intelligent Vehicle Applications Worldwide. In: Proceedings of the IEEE Intelligent Vehicles Symposium 2000, pp. 25–30 (2000)Google Scholar
  15. 15.
    Kuhnert, K.-D.: Software architecture of the Autonomous Mobile Outdoor Robot AMOR. In: Proceedings of the IEEE Intelligent Vehicles Symposium 2008, pp. 889–894 (2008)Google Scholar
  16. 16.
    Schilling, K.: Assistance Systems for the Control of Rovers. In: SICE Annual Conference, Tokyo (October 2008) Google Scholar
  17. 17.
    Wolf, D.F., Osório, F.S., Simões, E., Trindade Jr., O.: Robótica Inteligente: Da Simulação às Aplicações no Mundo Real [Tutorial]. In: de Leon F. de Carvalho, A.P., Kowaltowski, T. (eds.) JAI: Jornada de Atualização em Informática da SBC, vol. 1, pp. 279–330. SBC - Editora da PUC. RJ, Rio de Janeiro (2009)Google Scholar
  18. 18.
    Goebl, M., Althoff, M., Buss, M., Farber, G., Hecker, F., Heissing, B., Kraus, S., Nagel, R., Leon, F.P., Rattei, F., Russ, M., Schweitzer, M., Thuy, M., Wang, C., Wuensche, H.J.: Design and capabilities of the Munich Cognitive Automobile. In: IEEE Intelligent Vehicles Symposium 2008, pp. 1101–1107 (2008)Google Scholar
  19. 19.
    Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice Hall PTR, Upper Saddle River (1998)Google Scholar
  20. 20.
    Sales, D., Shinzato, P., Pessin, G., Wolf, D., Osório, F.: Vision-based Autonomous Navigation System using ANN and FSM Control. In: Proceedings of the IEEE Latin American Robotics Symposium (LARS), São Bernardo do Campo, Brazil (2010)Google Scholar
  21. 21.
    Giles, C.L., Horne, B.G., Lin, T.: Learning a class of large finite state machines with a recurrent neural network. Neural Networks 8(9), 1359–1365 (1995)CrossRefGoogle Scholar
  22. 22.
    Omlin, C.W., Giles, C.L.: Constructing Deterministic Finite-State Automata in Recurrent Neural Networks. Journal of the ACM 43(6), 937–972 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Frasconi, P., Gori, M., Maggini, M., Soda, G.: Representation of finite state automata in Recurrent Radial Basis Function networks. Machine Learning 23(1), 5–32 (1996)zbMATHGoogle Scholar
  24. 24.
    Cleeremans, A., Servan-Schreiber, D., Mcclelland, J.L.: Finite State Automata and Simple Recurrent Networks. Neural Computation 1(3), 372–381 (1989)CrossRefGoogle Scholar
  25. 25.
    KinectHacks Supporting the Kinect Hacking news and community, (accessed in: February 10, 2012)

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Daniel Sales
    • 1
  • Diogo Correa
    • 1
  • Fernando S. Osório
    • 1
  • Denis F. Wolf
    • 1
  1. 1.Mobile Robotics LabUniversity of São PauloSão CarlosBrazil

Personalised recommendations