Skip to main content
SpringerLink
Log in
Book cover

Reinforcement Learning of Bimanual Robot Skills pp 11–24Cite as

State of the Art

  • Chapter
  • First Online:

  • 994 Accesses

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 134))

Abstract

The recent trend of building more human-like robots and as using them in uncontrolled environments, presents several challenges that increase the complexity of task learning. It is required to have an integrated system capable of properly handling the kinematics and dynamics of the robot in the process of learning. Additionally, bimanual robots need to be configured in a way their arms are able to coordinate their motion while handling an object. Within this context, some key elements need to be taken into account, such as:

  • Robot kinematics, specially Inverse Kinematics (IK) for redundant serial robots.

  • Bimanual manipulation, in particular, the relative positioning of two robotic arms.

  • Compliant control and wrench estimation. Additionally to the kinematics aspect, we wanted to have a safe robot behavior also capable of detecting external disturbances while performing tasks.

In this chapter, we provide an overview of the key theoretical elements previous to our work, in order to accomplish the kinematic and dynamic control, taking into account workspace capabilities, kinematics and control.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Aghili, F.: Adaptive control of manipulators forming closed kinematic chain with inaccurate kinematic model. IEEE/ASME Trans. Mechatron. 18(5), 1544–1554 (2013)

    Article  Google Scholar 

  2. Alcocer, A., Robertsson, A., Valera, A., Johansson, R.: Force estimation and control in robot manipulators. In: Sasiadek, J., Duleba, I. (eds.) Robot Control 2003, pp. 31–36 (2004)

    Google Scholar 

  3. Antonelli, G.: Stability analysis for prioritized closed-loop inverse kinematic algorithms for redundant robotic systems. IEEE Trans. Robot. 25(5), 985–994 (2009)

    Article  Google Scholar 

  4. Assal, S.F.M., Watanabe, K., Izumi, K.: Neural network-based kinematic inversion of industrial redundant robots using cooperative fuzzy hint for the joint limits avoidance. IEEE/ASME Trans. Mechatron. 11(5), 593–603 (2006)

    Article  Google Scholar 

  5. Baillieul, J.: Kinematic programming alternatives for redundant manipulators. IEEE International Conference on Robotics and Automation (ICRA) 2, 722–728 (1985)

    Google Scholar 

  6. Burgard, W., Brock, O., Stachniss, C.: Safety Evaluation of Physical Human-Robot Interaction via Crash-Testing, pp. 1–352. MIT Press (2008)

    Google Scholar 

  7. Buss, S.R., Kim, J.-S.: Selectively damped least squares for inverse kinematics. J. Graph. Tools 10(3), 37–49 (2005)

    Article  Google Scholar 

  8. Chiaverini, S., Egeland, O., Kanestrom, R.K.: Achieving user-defined accuracy with damped least-squares inverse kinematics. In: IEEE Fifth International Conference on Advanced Robotics (ICAR), pp. 672–677 (1991)

    Google Scholar 

  9. Chiaverini, S., Oriolo, G., Walker, I.D.: Kinematically redundant manipulators. In: Handbook of Robotics. Springer (2008)

    Google Scholar 

  10. Chiaverini, S., Siciliano, B., Egeland, O.: Review of the damped least-squares inverse kinematics with experiments on an industrial robot manipulator. IEEE Trans. Control. Syst. Technol. 2(2), 123–134 (1994)

    Article  Google Scholar 

  11. Cline, A.K., Moler, C.B., Stewart, G.W., Wilkinson, J.H.: An estimate for the condition number of a matrix. SIAM J. Numer. Anal. 16(2), 368–375 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  12. Van Damme, M., Beyl, P., Vanderborght, B., Grosu, V., Van Ham, R., Vanderniepen, I., Matthys, A., Lefeber, D.: Estimating robot end-effector force from noisy actuator torque measurements. In IEEE International Conference on Robotics and Automation (ICRA), pp. 1108–1113 (2011)

    Google Scholar 

  13. Das, H., Slotine, J.-E., Sheridan, T.B.: Inverse kinematic algorithms for redundant systems. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 43–48 (1988)

    Google Scholar 

  14. Deo, A.S., Walker, I.D.: Minimum effort inverse kinematics for redundant manipulators. IEEE Trans. Robot. Autom. 13(5), 767–775 (1997)

    Article  Google Scholar 

  15. Ding, X., Fang, C.: A novel method of motion planning for an anthropomorphic arm based on movement primitives. IEEE/ASME Trans. Mechatron. 18(2), 624–636 (2013)

    Article  Google Scholar 

  16. Doty, K.L., Melchiorri, C., Schwartz, E.M., Bonivento, C.: Robot manipulability. IEEE Trans. Robot. Autom. 11(3), 462–468 (1995)

    Article  Google Scholar 

  17. Duffy, J.: The fallacy of modern hybrid control theory that is based on orthogonal complements of twist and wrench spaces. J. Robot. Syst. 139(2), 144–199 (1990)

    Google Scholar 

  18. Falco, P., Natale, C.: On the stability of closed-loop inverse kinematics algorithms for redundant robots. IEEE Trans. Robot. 27(4), 780–784 (2011)

    Article  Google Scholar 

  19. Funda, J., Taylor, R.H., Eldridge, B., Gomory, S., Gruben, K.G.: Constrained cartesian motion control for teleoperated surgical robots. IEEE Trans. Robot. Autom. 12(3), 453–465 (1996)

    Article  Google Scholar 

  20. Hyon, S.H., Hale, J.G., Cheng, G.: Full-body compliant human humanoid interaction: balancing in the presence of unknown external forces. IEEE Trans. Robot. 23(5), 884–898 (2007)

    Article  Google Scholar 

  21. Jiang, Z.-P., Wang, Y.: A converse lyapunov theorem for discrete-time systems with disturbances. Syst. Control Lett. 45(1), 49–58 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  22. Khatib, O.: A unified approach for motion and force control of robot manipulators: the operational space formulation. IEEE J. Robot. Autom. 3(1), 43–53 (1987)

    Article  Google Scholar 

  23. Kim, K.-Y., Song, H.-S., Suh, J.-W., Lee, J.-J.: A novel surgical manipulator with workspace-conversion ability for telesurgery. IEEE/ASME Trans. Mechatron. 18(1), 200–211 (2013)

    Article  Google Scholar 

  24. Klanke, S., Vijayakumar, S., Schaal, S.: A library for locally weighted projection regression. J. Mach. Learn. Res. 9, 623–626 (2008)

    MathSciNet  MATH  Google Scholar 

  25. Klein, C.A., Blaho, B.E.: Dexterity measures for the design and control of kinematically redundant manipulators. Int. J. Robot. Res. 6(2), 72–83 (1987)

    Article  Google Scholar 

  26. McLachlan, G.J., Peel, D.: Finite Mixture Models. Wiley (2000)

    Google Scholar 

  27. Mitrovic, D., Nagashima, S., Klanke, S., Matsubara, T., Vijayakumar, S.: Optimal feedback control for anthropomorphic manipulators. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 4143–4150 (2010)

    Google Scholar 

  28. Nakamura, Y.: Advanced Robotics: Redundancy and Optimization. Addison-Wesley Longman Publishing Co., Inc. (1990)

    Google Scholar 

  29. Nguyen-Tuong, D., Seeger, M., Peters, J.: Computed torque control with nonparametric regression models. In: American Control Conference, pp. 212–217 (2008)

    Google Scholar 

  30. Orin, D.E., Schrader, W.W.: Efficient computation of the jacobian for robot manipulators. Int. J. Robot. Res. 3(4), 66–75 (1984)

    Article  Google Scholar 

  31. Ott, C., Eiberger, O., Friedl, W., Bauml, B., Hillenbrand, U., Borst, C., Albu-Schaffer, A., Brunner, B., Hirschmuller, H., Kielhofer, S., Konietschke, R., Suppa, M., Wimbock, T., Zacharias, F., Hirzinger, G.: A humanoid two-arm system for dexterous manipulation. In: 6th IEEE-RAS International Conference on Humanoid Robots, pp. 276–283 (2006)

    Google Scholar 

  32. Park, I.-W., Lee, B.-J., Cho, S.-H., Hong, Y.-D., Kim, J.-H.: Laser-based kinematic calibration of robot manipulator using differential kinematics. IEEE/ASME Trans. Mechatron. 17(6), 1059–1067 (2012)

    Article  Google Scholar 

  33. Porta, J.M., Ros, L., Thomas, F.: Inverse kinematics by distance matrix completion. In: 12th International Workshop on Computational Kinematics, pp. 1–9 (2005)

    Google Scholar 

  34. Rabiner, L.R., McClellan, J.H., Parks, T.W.: FIR digital filter design techniques using weighted Chebyshev approximation. Proc. IEEE 63(4), 595–610 (1975)

    Article  Google Scholar 

  35. Ranjbaran, F., Angeles, J., Kecskeméthy, A.: On the kinematic conditioning of robotic manipulators. IEEE International Conference on Robotics and Automation (ICRA) 4, 3167–3172 (1996)

    Article  Google Scholar 

  36. Rao, R.S., Asaithambi, A., Agrawal, S.K.: Inverse kinematic solution of robot manipulators using interval analysis. J. Mech. Des. 120(1), 147–150 (1998)

    Article  Google Scholar 

  37. Ruiz de Angulo, V., Torras, C.: Self-calibration of a space robot. IEEE Trans. Neural Netw. 8(4), 951–963 (1997)

    Article  Google Scholar 

  38. Ruiz de Angulo, V., Torras, C.: Speeding up the learning of robot kinematics through function decomposition. IEEE Trans. Neural Netw. 16(6), 1504–1512 (2005)

    Article  Google Scholar 

  39. Ruiz de Angulo, V., Torras, C.: Learning inverse kinematics: reduced sampling through decomposition into virtual robots. IEEE Trans. Syst. Man Cybern. 38(6), 1571–1577 (2008)

    Article  Google Scholar 

  40. Sasaki, S.: Feasibility studies of kinematics problems in the case of a class of redundant manipulators. Robotica 13(03), 233–241 (1995)

    Article  Google Scholar 

  41. Siciliano, B.: A closed-loop inverse kinematic scheme for on-line joint-based robot control. Robotica 8(03), 231–243 (1990)

    Article  Google Scholar 

  42. Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G.: Robotics: Modelling, Planning and Control. Springer Science & Business Media (2010)

    Google Scholar 

  43. Singh, G.K., Claassens, J.: An analytical solution for the inverse kinematics of a redundant 7-DoF manipulator with link offsets. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2976–2982 (2010)

    Google Scholar 

  44. Smith, C., Karayiannidis, Y., Nalpantidis, L., Gratal, X., Qi, P., Dimarogonas, D.V., Kragic, D.: Dual arm manipulation–a survey. Robot. Auton. Syst. 60(10), 1340–1353 (2012)

    Article  Google Scholar 

  45. Tamei, T., Shibata, T.: Fast reinforcement learning for three-dimensional kinetic human-robot cooperation with an EMG-to-activation model. Adv. Robot. 25(5), 563–580 (2011)

    Article  Google Scholar 

  46. Townsend, W.T., Salisbury, J.K.: Mechanical Design for Whole-Arm Manipulation, pp. 153–164. Springer, Berlin (1993)

    Chapter  Google Scholar 

  47. Tucker, M., Perreira, N.D.: Generalized inverses for robotic manipulators. Mech. Mach. Theory 22(6), 507–514 (1987)

    Article  Google Scholar 

  48. Ulbrich, S., Ruiz de Angulo, V., Asfour, T., Torras, C., Dillmann, R.: Rapid learning of humanoid body schemas with kinematic bezier maps. In: 9th IEEE-RAS International Conference on Humanoid Robots, pp. 431–438 (2009)

    Google Scholar 

  49. Ulbrich, S., Ruiz De Angulo, V., Asfour, T., Torras, C., Dillmann, R.: General robot kinematics decomposition without intermediate markers. IEEE Trans. Neural Netw. Learn. Syst. 23(4), 620–630 (2012)

    Article  Google Scholar 

  50. Ulbrich, S., Ruiz de Angulo, V., Asfour, T., Torras, C., Dillmann, R.: Kinematic bezier maps. IEEE Trans. Syst. Man Cybern. 42(4), 1215–1230 (2012)

    Article  Google Scholar 

  51. Vahrenkamp, N., Przybylski, M., Asfour, T., Dillmann, R.: Bimanual grasp planning. In: IEEE-RAS 11th International Conference on Humanoid Robots (Humanoids), pp. 493–499 (2011)

    Google Scholar 

  52. Whitney, D.E.: Resolved motion rate control of manipulators and human prostheses. IEEE Trans. Man Mach. Syst. 10(2), 47–53 (1969)

    Article  MathSciNet  Google Scholar 

  53. Wolovich, W.A., Elliott, H.: A computational technique for inverse kinematics. In: IEEE Conference on Decision and Control, pp. 1359–1363 (1984)

    Google Scholar 

  54. Yoshikawa, T.: Analysis and control of robot manipulators with redundancy. In: Robotics Research: The First International Symposium, pp. 735–747. MIT Press Cambridge, MA (1984)

    Google Scholar 

  55. Yoshikawa, T.: Dynamic manipulability of robot manipulators. IEEE International Conference on Robotics and Automation 2, 1033–1038 (1985)

    Google Scholar 

  56. Zacharias, F., Borst, C., Hirzinger, G.: Capturing robot workspace structure: representing robot capabilities. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3229–3236 (2007)

    Google Scholar 

  57. Zacharias, F., Leidner, D., Schmidt, F., Borst, C., Hirzinger, G.: Exploiting structure in two-armed manipulation tasks for humanoid robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5446–5452 (2010)

    Google Scholar 

  58. Zanchettin, A.M., Ceriani, N.M., Rocco, P., Ding, H., Matthias, B.: Safety in human-robot collaborative manufacturing environments: metrics and control. IEEE Trans. Autom. Sci. Eng. 13(2), 882–893 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut de Robòtica i Informàtica Industrial (UPC-CSIC), Barcelona, Spain

    Adrià Colomé & Carme Torras

Authors
  1. Adrià Colomé
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carme Torras
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrià Colomé .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Colomé, A., Torras, C. (2020). State of the Art. In: Reinforcement Learning of Bimanual Robot Skills. Springer Tracts in Advanced Robotics, vol 134. Springer, Cham. https://doi.org/10.1007/978-3-030-26326-3_2

Download citation

Publish with us

Policies and ethics

Search

Navigation