ROS Framework for Perception and Dual-Arm Manipulation in Unstructured Environments

  • Delia SepúlvedaEmail author
  • Roemi Fernández
  • Eduardo Navas
  • Pablo González-de-Santos
  • Manuel Armada
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1093)


In a near future, robotic systems are expected to be able to confront more complex tasks in challenging scenarios. In this context, intelligent perception and dual-arm robotic manipulation capabilities are crucial for improving the autonomy and reliability of these systems. This paper addresses the development of an experimental platform conceived to facilitate the design and assessment of new perception and dual-arm control algorithms in unstructured environments. The proposed testbed is composed of a dual-arm robotic configuration endowed with a visual perception system and a simulation and control platform implemented in ROS (Robot Operating System). The robotic configuration consists of two manipulator arms of 6-DOF (Kinova MICO™) with brushless DC actuators controlled directly through PID controllers, whereas the perception system is formed by a high resolution RGB camera and a Time-of-Flight camera. ROS provides an open source collection of software frameworks, which simplify the task of creating complex and robust robot behaviours across a wide variety of robotic systems. The proposed approach will enable the easy testing and debugging of new applications with zero-risk damage to the real equipment.


Robot Operating System (ROS) Dual-arm robot manipulation Intelligent perception 



The research leading to these results has received funding from:

(i) FEDER/Ministerio de Ciencia, Innovación y Universidades – Agencia Estatal de Investigación/Proyecto ROBOCROP (DPI2017-84253-C2-1-R)

(ii) RoboCity2030-DIH-CM, Madrid Robotics Digital Innovation Hub, S2018/NMT-4331, funded by “Programas de Actividades I + D en la Comunidad de Madrid” and cofunded by Structural Funds of the EU.


  1. 1.
    Makris, S., Tsarouchi, P., Surdilovic, D., Krüger, J.: Intuitive dual arm robot programming for assembly operations. CIRP Ann. 63(1), 13–16 (2014)CrossRefGoogle Scholar
  2. 2.
    Zhao, Y., Gong, L., Liu, C., Huang, Y.: Dual-arm robot design and testing for harvesting tomato in greenhouse. IFAC-PapersOnLine 49(16), 161–165 (2016)CrossRefGoogle Scholar
  3. 3.
    Korayem, M.H., Shafei, A.M., Seidi, E.: Symbolic derivation of governing equations for dual-arm mobile manipulators used in fruit-picking and the pruning of tall trees. Comput. Electron. Agric. 105, 95–102 (2014)CrossRefGoogle Scholar
  4. 4.
    Wu, Q., Li, M., Qi, X., Hu, Y., Li, B., Zhang, J.: Coordinated control of a dual-arm robot for surgical instrument sorting tasks. Robot. Auton. Syst. 112, 1–12 (2019)CrossRefGoogle Scholar
  5. 5.
    Attia, M., Hossny, M., Nahavandi, S., Dalvand, M., Asadi, H.: Towards trusted autonomous surgical robots. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics, pp. 4083–4088 (2019)Google Scholar
  6. 6.
    Fleischer, H., Drews, R.R., Janson, J., Chinna Patlolla, B.R., Chu, X., Klos, M., Thurow, K.: Application of a dual-arm robot in complex sample preparation and measurement processes. J. Lab. Autom. 21(5), 671–681 (2016)CrossRefGoogle Scholar
  7. 7.
    Bustos, P., García-Varea, I., Martínez-Gómez, J., Mateos, J., Rodríguez-Ruiz, L., Sánchez, A.: Loki, a mobile manipulator for social robotics. In: Workshop of Physical Agents, pp. 1–8 (2012)Google Scholar
  8. 8.
    Quigley, M., Berkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.: ROS: an open-source robot operating system. In: Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan (2009)Google Scholar
  9. 9.
    MoveIt! Motion Planning Framework. Accessed 22 Apr 2019
  10. 10.
    Robotics company—Robotic assistive technology—Kinova. Accessed 22 Apr 2019
  11. 11.
    Campeau-Lecours, A., Lamontagne, H., Latour, S., Fauteux, P., Maheu, V., Boucher, F., L’Ecuyer, L.-J.C.: Kinova modular robot arms for service robotics applications. In: Rapid Automation. IGI Global (2019)Google Scholar
  12. 12.
    Fernández, R., Salinas, C., Montes, H., Sarria, J.: Multisensory system for fruit harvesting robots. Experimental testing in natural scenarios and with different kinds of crops. Sensors 14(12), 23885–23904 (2014)CrossRefGoogle Scholar
  13. 13.
    MESA Imaging SR4000/SR4500 User Manual Contents. Accessed 22 Apr 2019
  14. 14.
    Prosilica GC2450—5.0 Megapixel Sony ICX625 CCD sensor - Allied Vision. Accessed 22 Apr 2019
  15. 15.
    Image Processing Toolbox – MATLAB. Accessed 22 Apr 2019
  16. 16.
    Robotics System Toolbox - MATLAB®; Simulink. Accessed 22 Apr 2019
  17. 17.
    Martinez, A., Fernández, E.: Learning ROS for Robotics Programming. Packt Publishing, Birmingham (2013)Google Scholar
  18. 18.
    Salinas, C., Fernández, R., Montes, H., Armada, M.: A new approach for combining time-of-flight and RGB cameras based on depth-dependent planar projective transformations. Sensors 15(9), 24615–24643 (2015)CrossRefGoogle Scholar
  19. 19.
    Taylor, Z.: Accessed 22 Apr 2019
  20. 20.
    Concepts | MoveIt!. Accessed 22 Apr 2019

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Delia Sepúlveda
    • 1
    Email author
  • Roemi Fernández
    • 1
  • Eduardo Navas
    • 1
  • Pablo González-de-Santos
    • 1
  • Manuel Armada
    • 1
  1. 1.Centre for Automation and Robotics (UPM-CSIC)Spanish National Research CouncilArganda del Rey, MadridSpain

Personalised recommendations