Journal of Intelligent & Robotic Systems

, Volume 95, Issue 1, pp 47–60 | Cite as

Adaptive Robust Control for Bimanual Cooperative Contact Teleoperation with Relative Jacobian Matrix

  • Zhenyu Lu
  • Panfeng HuangEmail author
  • Ruizhou Cheng
  • Zhengxiong Liu
  • Zhongjie Meng


Due to the time delay in bilateral teleoperation, human reaction lags behind changes in the object state and the operating environment, which cannot solve unexpected accidents. An adaptive robust control strategy is proposed in this paper to address this problem. Inspired by human operational habits, two slaves are first divided into the dominant slave and the non-dominant slave with different functions. The non-dominant slave helps the dominant slave in humanoid style, as measured by relative Jacobian matrix. Then, the slaves’ reactions are divided into three stages: initial stage, adjustment stage, and recovery stage. In the second stage, the non-dominant slave helps the dominant slave according to its current state, and then they both track the master’s state in the third stage. The controllers of the two slaves are also designed differently. The dominant slave follows the dominant hand’s motions. Thus, an adaptive method is used based on RBF-neural networks to suppress uncertain system dynamics and forces. The non-dominant slave controller is designed in three stages consisting of three parts: the relative term, designed for collaborative motions; the shared term for sharing control signals from the dominant side; and the independent term for processing local disturbance. Finally, numerical experiments show the effectiveness of the proposed teleoperation architecture.


Adaptive control Relative Jacobian matrix RBF-neural networks Teleoperation Contact force 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This work was supported by the National Science Fund for Distinguished Young Scholars of China Grant No. 61725303 and the National Natural Science Foundation of China Grant No. 61773317 and the Fundamental Research Funds for the Central Universities Grant No. 3102016BJJ03.


  1. 1.
    Wilde, M., Chua, Z.K., Fleischner, A.: Effects of multivantage point systems on the teleoperation of spacecraft docking. IEEE Trans. Hum.-Mach. Syst. 44, 200–210 (2014)CrossRefGoogle Scholar
  2. 2.
    Chan, L.P., Naghdy, F., Stirling, D.: Position and force tracking for non-linear haptic telemanipulator under varying delays with an improved extended active observer. Robot. Auton. Syst. 75, 145–160 (2016)CrossRefGoogle Scholar
  3. 3.
    Schorr, S.B., Quek, Z.F., Nisky, I., Provancher, W.R., Okamura, A.M.: Tactor-induced skin stretch as a sensory substitution method in teleoperated palpation. IEEE Trans. Hum.-Mach. Syst. 45, 714–726 (2015)CrossRefGoogle Scholar
  4. 4.
    Rank, M., Shi, Z., Müller, H.J., Hirche, S.: Predictive communication quality control in haptic teleoperation with time delay and packet loss. IEEE Trans. Hum.-Mach. Syst. 46, 581–592 (2016)CrossRefGoogle Scholar
  5. 5.
    Guo, J., Guo, S.X., We, X., Gao, Q.: A Novel tele-operation controller for wireless microrobots in-pipe with hybrid motion. Robot. Auton. Syst. 76, 68–79 (2016)CrossRefGoogle Scholar
  6. 6.
    Takhmar, A., Polushin, I.G., Talasaz, A., Patel, R.V.: Cooperative teleoperation with projection-based force reflection for MIS. IEEE Trans. Control Syst. Technol. 23, 1411–1426 (2015)CrossRefGoogle Scholar
  7. 7.
    Hernansanz, A., Casals, A., Amat, J.: A multi-robot cooperation strategy for dexterous task oriented teleoperation. Robot. Auton. Syst. 68, 156–172 (2015)CrossRefGoogle Scholar
  8. 8.
    Lu, Z.Y., Huang, P.F., Liu, Z.X.: Predictive approach for sensorless bimanual teleoperation under random time delays with adaptive fuzzy control. IEEE Trans. Ind. Electron. 65(3), 2439–2448 (2018)CrossRefGoogle Scholar
  9. 9.
    Lu, Z.Y., Huang, P.F., Dai, P., Liu, Z.X., Meng, Z.J.: Enhanced transparency dual-user shared control teleoperation architecture with multiple adaptive dominance factors. Int. J. Control. Autom. Syst. 15, 2301–2312 (2017)CrossRefGoogle Scholar
  10. 10.
    Khademian, B., Hashtrudi-Zaad, K.: Dual-user teleoperation systems: new multilateral shared control architecture and kinesthetic performance measures. IEEE/ASME Trans. Mechatron. 17, 895–906 (2012)CrossRefGoogle Scholar
  11. 11.
    Khademian, B., Hashtrudi-Zaad, K.: A framework for unconditional stability analysis of multimaster /multislave teleoperation systems. IEEE Trans. Robot. 29, 684–694 (2013)CrossRefGoogle Scholar
  12. 12.
    Razi, K., Hashtrudi-Zaad, K.: Analysis of coupled stability in multilateral dual-user teleoperation systems. IEEE Trans. Robot. 30, 631–641 (2014)CrossRefGoogle Scholar
  13. 13.
    Shahbazi, M., Talebi, H.A., Atashzar, S.F., Towhidkhah, F., Patel, R.V., Shojaei, S.: A novel shared structure for dual user systems with unknown time-delay utilizing adaptive impedance control. In: IEEE international conference on robotics and automation, pp 2124–2129 (2011)Google Scholar
  14. 14.
    Malysz, P., Sirouspour, S.: Task performance evaluation of asymmetric semiautonomous teleoperation of mobile twin-arm robotic manipulators. IEEE Trans. Haptic 6, 484–495 (2013)CrossRefGoogle Scholar
  15. 15.
    Cheung, Y., Chung, J.H.: Semi-autonomous control of single-master multi-slave teleoperation of heterogeneous robots for multi-task multi-target pairing. Int. J. Control. Autom. 4, 1–17 (2011)Google Scholar
  16. 16.
    Hokayem, F., Stipanović, D.M., Spong, M.W.: Semiautonomous control of multiple networked Lagrangian systems. Int. J. Robust Nonlinear Control 19, 2040–2055 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Sirouspour, S., Setoodeh, P.: Multi-operator/multi-robot teleoperation: an adaptive nonlinear control approach. In: IEEE international conference on intelligent robots and systems, pp 1576–1581 (2005)Google Scholar
  18. 18.
    Arcara, P., Melchiorri, C.: Control schemes for teleoperation with time delay: a comparative study. Robot. Auton. Syst. 38, 49–64 (2002)CrossRefzbMATHGoogle Scholar
  19. 19.
    Nuño, E., Sarras, I., Basañez, L., Kinnaert, M., Slawinski, E.: Control of teleoperators with joint flexibility, uncertain parameters and time-delays. Robot. Auton. Syst. 62, 1691–1701 (2014)CrossRefGoogle Scholar
  20. 20.
    Lee, D., Spong, M.W.: Bilateral teleoperation of multiple cooperative robots over delayed communication networks: theory. In: IEEE international conference on robotics and automation, pp 360–365 (2005)Google Scholar
  21. 21.
    Lee, D., Martinez, P.O., Spong, M.W.: Bilateral teleoperation of multiple cooperative robots over delayed communication networks: applications. In: IEEE international conference on robotics and automation, pp 366–371 (2005)Google Scholar
  22. 22.
    Li, Z., Su, C.Y.: Neural-adaptive control of single-master multiple-slaves teleoperation for coordinated multiple mobile manipulators with time-varying communication delays and input uncertainties. IEEE Trans. Neural Netw. Learn. Syst. 24, 1400–1413 (2013)CrossRefGoogle Scholar
  23. 23.
    Li, Z., Xia, Y., Sun, F.: Adaptive fuzzy control for multilateral cooperative teleoperation of multiple robotic manipulators under random network-induced delays. IEEE Trans. Fuzzy Syst. 22, 437–450 (2014)CrossRefGoogle Scholar
  24. 24.
    Chen, Z., Pan, Y.J., Gu, J.: Integrated adaptive robust control for multilateral teleoperation systems under arbitrary time delays. Int. J. Robust Nonlinear Control 26, 2708–2728 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Li, J., Tavakoli, M., Huang, Q.: Stability of cooperative teleoperation using haptic devices with complementary degrees of freedom. IET Control Theory Appl. 8, 1062–1070 (2014)CrossRefGoogle Scholar
  26. 26.
    Kanno, T., Yokokohji, Y.: Multilateral teleoperation control over time-delayed computer networks using wave variables. In: IEEE international conference on Haptics symposium, pp 125–131 (2012)Google Scholar
  27. 27.
    Elhajj, I.H., Goradia, A., Xi, N., Kit, C.M., Liu, Y.H., Fukuda, T.: Design and analysis of internet-based tele-coordinated multi-robot systems. Auton. Robot. 15, 237–254 (2003)CrossRefGoogle Scholar
  28. 28.
    Ma, L., Yan, J., Zhao, J., Chen, Z., Cai, H.: Teleoperation system of internet-based multi-operator multi-mobile-manipulator. In: IEEE international conference on electrical and control engineering, pp 2236–2240 (2010)Google Scholar
  29. 29.
    Huang, P.F., Dai, P., Lu, Z.Y., Liu, Z.X.: Asymmetric wave variable compensation method in dual-master-dual-slave multilateral teleoperation system. Mechatronics 49, 1–10 (2018)CrossRefGoogle Scholar
  30. 30.
    Goodrich, M.A., Crandall, J.W., Barakova, E.: Teleoperation and beyond for assistive humanoid robots. Rev. Hum. Factors Ergon. 9, 175–226 (2013)CrossRefGoogle Scholar
  31. 31.
    Phillips-Grafflin, C., Suay, H.B., Mainprice, J., et al.: From autonomy to cooperative traded control of humanoid manipulation tasks with unreliable communication. J. Intell. Robot. Syst. 82, 341–361 (2016)CrossRefGoogle Scholar
  32. 32.
    Lee, J., Chang, P.H., Jamisola, R.S.: Relative impedance control for dual-arm robots performing asymmetric bimanual tasks. IEEE Trans. Ind. Electron. 61, 3786–3796 (2014)CrossRefGoogle Scholar
  33. 33.
    Balakrishnan, R., Hinckley, K.: Symmetric bimanual interaction. In: SIGCHi conference on human factors in computing systems, pp 33–40 (2000)Google Scholar
  34. 34.
    Moody, J., Darken, C.J.: Fast learning in networks of locally-tuned processing units. Neural Comput. 1, 281–294 (1989)CrossRefGoogle Scholar
  35. 35.
    Er, M. J., Wu, S., Lu, J.: Face recognition with radial basis function (RBF) neural networks. IEEE Trans. Neural Netw. 13, 697–710 (2002)CrossRefGoogle Scholar
  36. 36.
    Pierre, R.B.: Estimation of angular velocity and acceleration from shaft encoder measurements. In: International conference on robotics and automation, pp 585–589 (1992)Google Scholar
  37. 37.
    Violaine, C., Martin, L., Amarantini, D.: Static optimal estimation of joint accelerations for inverse dynamics problem solution. J. Biomech. 35, 1507–1513 (2002)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  • Zhenyu Lu
    • 1
    • 2
  • Panfeng Huang
    • 1
    • 2
    Email author
  • Ruizhou Cheng
    • 1
    • 2
  • Zhengxiong Liu
    • 1
    • 2
  • Zhongjie Meng
    • 1
    • 2
  1. 1.Research Center for Intelligent Robotics, School of AstronauticsNorthwestern Polytechnical UniversityXi’anChina
  2. 2.School of Astronautics and National Key Laboratory of Aerospace Flight DynamicsNorthwestern Polytechnical UniversityXi’anChina

Personalised recommendations