Autonomous Interactive Object Manipulation and Navigation Capabilities for an Intelligent Wheelchair

  • Nima ShafiiEmail author
  • P. C. M. A. Farias
  • Ivo Sousa
  • Heber Sobreira
  • Luis Paulo Reis
  • Antonio Paulo Moreira
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10423)


This paper aims to develop grasping and manipulation capability along with autonomous navigation and localization in a wheelchair-mounted robotic arm to serve patients. Since the human daily environment is dynamically varied, it is not possible to enable the robot to know all the objects that would be grasped. We present an approach to enable the robot to detect, grasp and manipulate unknown objects. We propose an approach to construct the local reference frame that can estimate the object pose for detecting the grasp pose of an object. The main objective of this paper is to present the grasping and manipulation approach along with a navigating and localization method that can be performed in the human daily environment. A grid map and a match algorithm is used to enable the wheelchair to localize itself using a low-power computer. The experimental results show that the robot can manipulate multiple objects and can localize itself with great accuracy.



This work is financed by the ERDF European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation - COMPETE 2020 Programme, and by National Funds through the FCT Fundao para a Ciłncia e a Tecnologia (Portuguese Foundation for Science and Technology) within project POCI-01-0145-FEDER-006961. P.C.M.A. Farias acknowledge support from CNPq/CsF PDE 233517/2014-6 for providing a scholarship.


  1. 1.
    Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesisa survey. IEEE Trans. Robot. 30(2), 289–309 (2014)CrossRefGoogle Scholar
  2. 2.
    Censi, A.: An ICP variant using a point-to-line metric. In: IEEE International Conference on Robotics and Automation, ICRA 2008, pp. 19–25. IEEE (2008)Google Scholar
  3. 3.
    Holz, D., Holzer, S., Rusu, R.B., Behnke, S.: Real-time plane segmentation using RGB-D cameras. In: Röfer, T., Mayer, N.M., Savage, J., Saranlı, U. (eds.) RoboCup 2011. LNCS, vol. 7416, pp. 306–317. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-32060-6_26CrossRefGoogle Scholar
  4. 4.
    Hsiao, K., Chitta, S., Ciocarlie, M., Jones, E.G.: Contact-reactive grasping of objects with partial shape information. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1228–1235. IEEE (2010)Google Scholar
  5. 5.
    Kasaei, S.H., Shafii, N., Lopes, L.S., Tomé, A.M.: Object learning and grasping capabilities for robotic home assistants. LNCS, vol. 9776. Springer, Cham (2016)Google Scholar
  6. 6.
    Kim, D.J., Wang, Z., Paperno, N., Behal, A.: System design and implementation of UCF-MANUS an intelligent assistive robotic manipulator. IEEE/ASME Trans. Mechatron. 19(1), 225–237 (2014)CrossRefGoogle Scholar
  7. 7.
    Ktistakis, I.P., Bourbakis, N.G.: A survey on robotic wheelchairs mounted with robotic arms. In: 2015 National Aerospace and Electronics Conference (NAECON), pp. 258–262. IEEE (2015)Google Scholar
  8. 8.
    Lauer, M., Lange, S., Riedmiller, M.: Calculating the perfect match: an efficient and accurate approach for robot self-localization. In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds.) RoboCup 2005. LNCS, vol. 4020, pp. 142–153. Springer, Heidelberg (2006). doi: 10.1007/11780519_13CrossRefGoogle Scholar
  9. 9.
    Library, P.C.: Plane model segmentation documentation (2017). Accessed 10 Feb 2017
  10. 10.
    Maheu, V., Archambault, P.S., Frappier, J., Routhier, F.: Evaluation of the JACO robotic arm: clinico-economic study for powered wheelchair users with upper-extremity disabilities. In: 2011 IEEE International Conference on Rehabilitation Robotics (ICORR), pp. 1–5. IEEE (2011)Google Scholar
  11. 11.
    Miller, A.T., Allen, P.K.: Graspit! a versatile simulator for robotic grasping. IEEE Robot. Autom. Mag. 11(4), 110–122 (2004)CrossRefGoogle Scholar
  12. 12.
    Pinto, A.C.P.: Advanced Mobile Manipulation for Logistics in Hospitals or Laboratories (2016)Google Scholar
  13. 13.
    Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3, p. 5 (2009)Google Scholar
  14. 14.
    Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–4. IEEE (2011)Google Scholar
  15. 15.
    Sobreira, H., Pinto, M., Moreira, A.P., Costa, P.G., Lima, J.: Robust robot localization based on the perfect match algorithm. In: Moreira, A.P., Matos, A., Veiga, G. (eds.) CONTROLO 2014. LNEE, vol. 321, pp. 607–616. Springer, Cham (2015). doi: 10.1007/978-3-319-10380-8_58CrossRefGoogle Scholar
  16. 16.
    Sobreira, H., Rocha, L., Costa, C., Lima, J., Costa, P., Moreira, A.P.: 2D cloud template matching-a comparison between iterative closest point and perfect match. In: 2016 International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 53–59. IEEE (2016)Google Scholar
  17. 17.
    Tanaka, H., Sumi, Y., Matsumoto, Y.: Assistive robotic arm autonomously bringing a cup to the mouth by face recognition. In: 2010 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), pp. 34–39. IEEE (2010)Google Scholar
  18. 18.
    Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. Intelligent Robotics and Autonomous Agents. MIT Press, Cambridge (2005)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Nima Shafii
    • 1
    Email author
  • P. C. M. A. Farias
    • 1
    • 2
  • Ivo Sousa
    • 1
  • Heber Sobreira
    • 1
  • Luis Paulo Reis
    • 3
  • Antonio Paulo Moreira
    • 1
  1. 1.INESC Technology and Science, Faculty of EngineeringUniversity of PortoPortoPortugal
  2. 2.Polytechnic SchoolFederal University of BahiaSalvadorBrazil
  3. 3.Departamento de Sistemas de InformaoUniversidade do MinhoGuimaresPortugal

Personalised recommendations