Human Multi-Robot Physical Interaction: a Distributed Framework

Abstract

The objective of this paper is to devise a general framework to allow a human operator to physically interact with an object manipulated by a multi-manipulator system in a distributed setting. A two layer solution is devised. In detail, at the top layer an arbitrary virtual dynamics is considered for the object with the virtual input chosen as the solution of an optimal Linear Quadratic Tracking (LQT) problem. In this formulation, both the human and robots’ intentions are taken into account, being the former online estimated by Recursive Least Squares (RLS) technique. The output of this layer is a desired trajectory of the object which is the input of the bottom layer and from which desired trajectories for the robot end effectors are computed based on the closed-chain constraints. Each robot, then, implements a time-varying gain adaptive control law so as to take into account model uncertainty and internal wrenches that inevitably raise due to synchronization errors and dynamic and kinematic uncertainties. Remarkably, the overall solution is devised in a distributed setting by resorting to a leader-follower approach and distributed observers. Simulations with three 6-DOFs serial chain manipulators mounted on mobile platforms corroborate the theoretical findings.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Lacevic, B., Rocco, P., Zanchettin, A.M.: Safety assessment and control of robotic manipulators using danger field. IEEE Trans. Robot. 29(5), 1257–1270 (2013)

    Article  Google Scholar 

  2. 2.

    Lippi, M., Marino, A.: Safety in human-multi robot collaborative scenarios: a trajectory scaling approach. IFAC-PapersOnLine 51(22), 190–196 (2018)

    Article  Google Scholar 

  3. 3.

    Tsumugiwa, T., Yokogawa, R., Hara, K.: Variable impedance control based on estimation of human arm stiffness for human-robot cooperative calligraphic task. In: IEEE Int. Conf. Robot. Autom., vol. 1, pp. 644–650. https://ieeexplore.ieee.org/document/1013431 (2002)

  4. 4.

    Dimeas, F., Aspragathos, N.: Online Stability in Human-Robot Cooperation with Admittance Control. IEEE Transactions on Haptics 9(2), 267–278 (2016)

    Article  Google Scholar 

  5. 5.

    Li, Y., Ge, S.S.: Human–robot collaboration based on motion intention estimation. IEEE/ASME Trans. Mechatronics 19(3), 1007–1014 (2014)

    Article  Google Scholar 

  6. 6.

    Corteville, B., Aertbelien, E., Bruyninckx, H., De Schutter J., Van Brussel, H.: Human-inspired robot assistant for fast point-to-point movements. In: Proceedings 2007 IEEE International Conference on Robotics and Automation, pp 3639–3644. Roma, Italy (2007)

  7. 7.

    Losey, D.P., McDonald, C.G., Battaglia, E., O’Malley, M.K.: A review of intent detection, arbitration, and communication aspects of shared control for physical human–robot interaction. Applied Mechanics Reviews 70(1), 010804 (2018)

    Article  Google Scholar 

  8. 8.

    Musić, S., Hirche, S.: Control sharing in human-robot team interaction. Annu. Rev. Control. 44, 342–354 (2017)

    Article  Google Scholar 

  9. 9.

    Masone, C., Mohammadi, M., Robuffo Giordano, P., Franchi, A.: Shared planning and control for mobile robots with integral haptic feedback. Int. J. Robot. Res. 37(11), 1395–1420 (2018)

    Article  Google Scholar 

  10. 10.

    Li, Y., Tee, K.P., Chan, W.L., Yan, R., Chua, Y., Limbu, D.K.: Continuous role adaptation for human–robot shared control. IEEE Trans Robot. 31(3), 672–681 (2015)

    Article  Google Scholar 

  11. 11.

    Losey, D.P., O’Malley, M.K.: Trajectory deformations from physical human–robot interaction. IEEE Trans Robot. 34(1), 126–138 (2018)

    Article  Google Scholar 

  12. 12.

    Mörtl, A., Lawitzky, M., Kucukyilmaz, A., Sezgin, M., Basdogan, C., Hirche, S.: The role of roles: physical cooperation between humans and robots. Int. J. Robot. Res. 31(13), 1656–1674 (2012)

    Article  Google Scholar 

  13. 13.

    Medina, J.R., Lorenz, T., Hirche, S.: Synthesizing anticipatory haptic assistance considering human behavior uncertainty. IEEE Trans. Robot. 31(1), 180–190 (2015)

    Article  Google Scholar 

  14. 14.

    Villani, V., Pini, F., Leali, F., Secchi, C.: Survey on human–robot collaboration in industrial settings: safety, intuitive interfaces and applications. Mechatronics 55, 248–266 (2018)

    Article  Google Scholar 

  15. 15.

    Lippi, M., Marino, A., Chiaverini, S.: A distributed approach to human multi-robot physical interaction. In: IEEE Int. Conf. on Systems, Man and Cybernetics, pp. 728–734 (2019)

  16. 16.

    Khatib, O.: A unified approach for motion and force control of robot manipulators: the operational space formulation. IEEE J. Robot. Autom. 3(1), 43–53 (1987)

    Article  Google Scholar 

  17. 17.

    Colbaugh, R., Seraji, H., Glass, K.: Direct adaptive impedance control of robot manipulators. J. Robot Syst. 10(2), 217–248 (1993)

    Article  Google Scholar 

  18. 18.

    Rahman, M.D.M., Ikeura, R., Mizutani, K.: Investigation of the impedance characteristic of human arm for development of robots to cooperate with humans. JSME Int. J. Series C Mechanical Syst., Machine Elem. and Manufacturing 45(2), 510–518 (2002). https://www.jstage.jst.go.jp/article/jsmec/45/2/45_2_510/_article/-char/en

    Google Scholar 

  19. 19.

    Burdet, E., Franklin, D.W., Milner, T.E.: Human Robotics: Neuromechanics and Motor Control. MIT Press (2013)

  20. 20.

    Erden, M.S., Billard, A.: Hand impedance measurements during interactive manual welding with a robot. IEEE Trans. Robot. 31(1), 168–179 (2015)

    Article  Google Scholar 

  21. 21.

    Artemiadis, P.K., Katsiaris, P.T., Liarokapis, M.V., Kyriakopoulos, K.J.: Human arm impedance: characterization and modeling in 3D space. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 3103–3108 (2010)

  22. 22.

    Modares, H., Ranatunga, I., Lewis, F.L., Popa, D.O.: Optimized assistive human–robot interaction using reinforcement learning. IEEE Trans. Cybern. 46(3), 655–667 (2016)

    Article  Google Scholar 

  23. 23.

    Marino, A.: Distributed adaptive control of networked cooperative mobile manipulators. IEEE Trans. Control Syst. Technol. 26(5), 1646–1660 (2018)

    Article  Google Scholar 

  24. 24.

    Marino, A., Pierri, F.: A two stage approach for distributed cooperative manipulation of an unknown object without explicit communication and unknown number of robots. Rob. Auton. Syst. 103, 122–133 (2018)

    Article  Google Scholar 

  25. 25.

    Mesbahi, M., Egerstedt, M.: Graph theoretic methods in multiagent networks. Princeton University Press (2010)

  26. 26.

    Selvaggio, M., Abi-Farraj, F., Pacchierotti, C., Giordano, P.R., Siciliano, B.: Haptic-based shared-control methods for a dual-arm system. IEEE Robot. Autom Lett. 3(4), 4249–4256 (2018)

    Article  Google Scholar 

  27. 27.

    Yoshikawa, T., Zheng, X.: Coordinated dynamic hybrid position/force control for multiple robot manipulators handling one constrained object. Int. J. Robot. Res. 12(3), 219–230 (1993)

    Article  Google Scholar 

  28. 28.

    De Luca, A., Mattone, R.: Sensorless robot collision detection and hybrid force/motion control. In: IEEE Int. Conf. Robot. Autom., pp. 999–1004 (2005)

  29. 29.

    Haykin, S.: Adaptive Filter Theory. Prentice-Hall, Inc. (1996)

  30. 30.

    Malik, O.P., Hope, G.S., Cheng, S.J.: Some issues on the practical use of recursive least squares identification in self-tuning control. Int. J. Control 53(5), 1021–1033 (1991)

    MathSciNet  Article  Google Scholar 

  31. 31.

    Rao Sripada, N., Grant Fisher, D.: Improved least squares identification. Int. J. Control 46 (6), 1889–1913 (1987)

    MathSciNet  Article  Google Scholar 

  32. 32.

    Bhole, A., Ficuciello, F., Mashayekhi, A., Strano, S., Terzo, M., Villani, L., Siciliano, B.: Online estimation of impedance parameters for a variable impedance controlled robotic manipulator. In: Advances in Italian Mechanism Science, pp 267–274. Springer International Publishing (2019)

  33. 33.

    Lewis, F.L., Vrabie, D., Syrmos, V.L.: Optimal Control. Wiley, New York (2012)

    Google Scholar 

  34. 34.

    Chen, F., Cao, Y., Ren, W.: Distributed average tracking of multiple time-varying reference signals with bounded derivatives. IEEE Trans. Autom Control 57(12), 3169–3174 (2012)

    MathSciNet  Article  Google Scholar 

  35. 35.

    Fu, J., Wang, J.: Observer-based finite-time coordinated tracking for general linear multi-agent systems. Automatica 66, 231–237 (2016)

    MathSciNet  Article  Google Scholar 

  36. 36.

    Golluccio, G., Gillini, G., Marino, A., Antonelli, G.: Robot dynamics identification: a reproducible comparison with experiments on the kinova jaco2. IEEE Robot. Autom Mag. (2020)

  37. 37.

    Lippi, M., Marino, A.: Cooperative object transportation by multiple ground and aerial vehicles: modeling and planning. In: IEEE Int. Conf. Robot. Autom., pp 1084–1090 (2018)

  38. 38.

    Li, Y., Ge, S.S.: Force tracking control for motion synchronization in human-robot collaboration. Robotica 34(6), 1260–1281 (2016)

    Article  Google Scholar 

  39. 39.

    Sun, K., Qiu, J., Karimi, H.R., Gao, H.: A novel finite-time control for nonstrict feedback saturated nonlinear systems with tracking error constraint. IEEE Trans Systems, Man, and Cybernetics: Systems, 1–12. https://ieeexplore.ieee.org/document/8944301 (2019)

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Martina Lippi.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by Dipartimento di Eccellenza granted to DIEI Department, University of Cassino and Southern Lazio and by H2020-ICT project CANOPIES-A Collaborative Paradigm for Human Workers and Multi-Robot Teams in Precision Agriculture Systems (Grant Agreement N. 101016906).

Appendix

Appendix

To prove Theorem 1, let us first derive the closed loop dynamics or robot i. By folding (45) in (1), it holds

$$ {\boldsymbol{M}}_{\!i}\dot{\boldsymbol{s}}_{i} = -{\boldsymbol{C}}_{i}\boldsymbol{s}_{i} - \boldsymbol{K}_{s}\boldsymbol{s}_{i} - {\varDelta} \boldsymbol{u}_{i} + \boldsymbol{h}_{i} + {\boldsymbol{Y}}_{\!\!i}(\boldsymbol{x}_{i}, \dot{\boldsymbol{x}}_{i},\boldsymbol{\rho}_{i}, \dot{\boldsymbol{\rho}}_{i} )\tilde{\boldsymbol{\pi}}_{i}\!\!\!\! $$
(48)

Now, let us consider the following Lyapunov function

$$ V=\frac{1}{2}\sum\limits_{i=1}^{N}\left( \boldsymbol{s}_{i}^{\mathrm{T}}{\boldsymbol{M}}_{i}\boldsymbol{s}_{i}+\frac{1}{k_{f}}\hat{\varDelta} \boldsymbol{u}_{f,i}^{\mathrm{T}}\hat{\varDelta} \boldsymbol{u}_{f,i}+\tilde{\boldsymbol{\pi}}_{i}^{\mathrm{T}}\boldsymbol{K}_{\pi}\tilde{\boldsymbol{\pi}}_{i}\right) $$
(49)

By virtue of (48) and Property 1, the time derivative of V is

$$ \begin{aligned} {}\dot{V} \! =& \! \sum\limits_{i=1}^{N}\!\Big(\boldsymbol{s}_{i}^{\mathrm{T}}{\boldsymbol{M}}_{i}\dot{\boldsymbol{s}}_{i} + \frac{1}{2}\boldsymbol{s}_{i}^{\mathrm{T}}\dot{{\boldsymbol{M}}}_{i}\boldsymbol{s}_{i} + \hat{\varDelta} \boldsymbol{u}_{f,i}^{\mathrm{T}}\tilde{\boldsymbol{h}}_{int,i} - \tilde{\boldsymbol{\pi}}_{i}^{\mathrm{T}}\boldsymbol{K}_{\pi}{\dot{\hat{\boldsymbol{\pi}}}}_{i}\Big)\\ =&\! \sum\limits_{i=1}^{N}\Big(-\boldsymbol{s}_{i}^{\mathrm{T}}\boldsymbol{K}_{s}\boldsymbol{s}_{i}+\frac{1}{2}(\dot{\boldsymbol{M}}_{i}-2{\boldsymbol{C}}_{i})\boldsymbol{s}_{i} +\boldsymbol{s}_{i}^{\mathrm{T}}(\boldsymbol{h}_{i}-{\varDelta} \boldsymbol{u}_{i})\\ &+\hat{\varDelta} \boldsymbol{u}_{f,i}^{\mathrm{T}}\tilde{\boldsymbol{h}}_{int,i} -\tilde{\boldsymbol{\pi}}_{i}^{\mathrm{T}}(\boldsymbol{K}_{\pi}{\dot{\hat{\boldsymbol{\pi}}}}_{i}-{\boldsymbol{Y}}_{i}^{\mathrm{T}}(\boldsymbol{x}_{i}, \dot{\boldsymbol{x}}_{i}, \boldsymbol{\rho}_{i}, \dot{\boldsymbol{\rho}}_{i} )\boldsymbol{s}_{i})\Big)\\ =&\!\sum\limits_{i=1}^{N}\!\Big(\! - \boldsymbol{s}_{i}^{\mathrm{T}}{\kern-.5pt}\boldsymbol{K}_{s}\boldsymbol{s}_{i} + \hat{\varDelta} \boldsymbol{u}_{f,i}^{\mathrm{T}}\tilde{\boldsymbol{h}}_{int,i} - \boldsymbol{s}_{i}^{\mathrm{T}}{\kern-.5pt}({\kern-.5pt}\boldsymbol{e}_{int,i} + k_{f}\!\hat{\varDelta} \boldsymbol{u}_{f,i}\! +\! \frac{1}{N}{~}^{i}\tilde{\boldsymbol{h}}_{h}{\kern-.5pt})\\ & -\kappa_{i}(t)\boldsymbol{s}_{i}^{\mathrm{T}}\boldsymbol{s}_{i} -\tilde{\boldsymbol{\pi}}_{i}^{\mathrm{T}}(\boldsymbol{K}_{\pi}{\dot{\hat{\boldsymbol{\pi}}}}_{i}-{\boldsymbol{Y}}_{i}^{\mathrm{T}}(\boldsymbol{x}_{i}, \dot{\boldsymbol{x}}_{i}, \boldsymbol{\rho}_{i}, \dot{\boldsymbol{\rho}}_{i} )\boldsymbol{s}_{i}) \Big)\\ \end{aligned} $$
(50)

Given the parameters update law in Eq. (47), (50) simplifies to

$$ \begin{aligned} \dot{V} =&\sum\limits_{i=1}^{N}\Big(-\boldsymbol{s}_{i}^{\mathrm{T}}\boldsymbol{K}_{s}\boldsymbol{s}_{i}+\hat{\varDelta} \boldsymbol{u}_{f,i}^{\mathrm{T}}\tilde{\boldsymbol{h}}_{int,i}-\kappa_{i}(t)\|\boldsymbol{s}_{i}\|^{2}\\ &-(\tilde{\boldsymbol{\zeta}}_{i}+\hat{\varDelta} \boldsymbol{u}_{f,i})^{\mathrm{T}}(\tilde{\boldsymbol{h}}_{int,i}+\hat{\varDelta} \boldsymbol{u}_{f,i})+\frac{1}{N}\boldsymbol{s}_{i}^{\mathrm{T}}{~}^{i}\tilde{\boldsymbol{h}}_{h} \Big)\\ \end{aligned} $$
(51)

Thus, by choosing

$$\kappa_{i}(t)>\frac{\|\tilde{\boldsymbol{\zeta}}_{i}\|}{\|\boldsymbol{s}_{i}\|^{2}} \|\tilde{\boldsymbol{h}}_{int,i}+\hat{\varDelta} \boldsymbol{u}_{f,i}\|$$

it holds

$$ \dot{V}\leq \sum\limits_{i=1}^{N}\big(-\boldsymbol{s}_{i}^{\mathrm{T}}\boldsymbol{K}_{s}\boldsymbol{s}_{i}-\hat{\varDelta} \boldsymbol{u}_{f,i}^{\mathrm{T}}\hat{\varDelta} \boldsymbol{u}_{f,i}+\frac{1}{N}\|\boldsymbol{s}_{i}^{\mathrm{T}}\|\|{}^{i}\tilde{\boldsymbol{h}}_{h}\|\big) $$

From Lemma 2, \(\|{}^{i}\tilde {\boldsymbol {h}}_{h}\|\) converges to the origin after a finite time Th; then after this time it holds

$$ \dot{V}\leq \sum\limits_{i=1}^{N}\big(-\boldsymbol{s}_{i}^{\mathrm{T}}\boldsymbol{K}_{s}\boldsymbol{s}_{i}-{~}^{i}\hat{\varDelta} \boldsymbol{u}_{f,i}^{\mathrm{T}}{~}^{i}\hat{\varDelta} \boldsymbol{u}_{f,i}\big) $$

which implies that \(\dot {V}\) is semi-negative definite and, consequently, that V is bounded. By leveraging the boundedness of V and, then, of si, \(\hat {\varDelta } \boldsymbol {u}_{f,i}\) and \({\tilde {\boldsymbol {\pi }}}_{i}\), it can be easily shown the \(\ddot V\) is bounded as well. Thus, by virtue of Barbalat’s lemma, \(\dot {V}\) is uniformly continuous and converges to the origin, as well as si and \({\hat {\varDelta } \boldsymbol {u}_{f,i}=k_{f}{\int \limits }_{t_{0}}^{t}{\tilde {\boldsymbol {h}}_{int,i}} d\tau }\) (and, therefore, by definition \(\tilde {\boldsymbol {h}}_{int,i}\)). The main implication of the latter is that, since because of Lemma 2, \(\tilde {\boldsymbol {h}}_{int,i}\) converges to the origin in finite time (that is the internal wrenches estimated via observer converges to the real one), then also eint, i, ∀i, converges to the origin.

In view of the expression of si in (43) and since \(\hat {\varDelta } \boldsymbol {u}_{f,i}\) converges to the origin, it follows

$$ \dot{\boldsymbol{e}}_{x,i}+k_{p}\boldsymbol{e}_{x,i} =-\hat{\varDelta} \boldsymbol{u}_{f,i} $$

which represents an asymptotically stable system (in the state variable ex, i) with vanishing input \(-\hat {\varDelta } \boldsymbol {u}_{f,i}\). Therefore, ex, i asymptotically converges to the origin. Based on the expression of ex, i in (42), it asymptotically holds

$$ ({~}^{i} \hat{\boldsymbol{x}}_{v}-\boldsymbol{x}_{i})+k_{c}{\int}_{t_{0}}^{t}\sum\limits_{j\in\mathcal{N}_{i}}(\boldsymbol{x}_{j}-\boldsymbol{x}_{i})d\tau\rightarrow\boldsymbol{0}_{p} $$
(52)

Let us now introduce the object trajectory estimate error \({~}^{i}\tilde {\boldsymbol {x}}_{v} = {\boldsymbol {x}}_{v} - {~}^{i} \hat {\boldsymbol {x}}_{v}\in \Re ^{p}\) and the object trajectory tracking error ev, i = xvxiRp. From (52), it asymptotically holds

$$ \begin{aligned} ({\boldsymbol{x}}_{v}-\boldsymbol{x}_{i})+k_{c}{\int}_{t_{0}}^{t}\sum\limits_{j\in\mathcal{N}_{i}}(\boldsymbol{x}_{j}-{\boldsymbol{x}}_{v}-\boldsymbol{x}_{i}+{\boldsymbol{x}}_{v})d\tau&=-{~}^{i}\tilde{\boldsymbol{x}}_{v}\\ \end{aligned} $$

which can be rewritten as

$$ \begin{aligned} \boldsymbol{e}_{v,i}+k_{c}{\int}_{t_{0}}^{t}\sum\limits_{j\in\mathcal{N}_{i}}(-\boldsymbol{e}_{v,j}+\boldsymbol{e}_{v,i})d\tau&=-{~}^{i}\tilde{\boldsymbol{x}}_{v}\\ \end{aligned} $$
(53)

By denoting with \(\tilde {\boldsymbol {x}}_{v}\in \Re ^{Np}\) and evRNp the stacked vectors of the errors \({~}^{i}\tilde {\boldsymbol {x}}_{v}\) and ev, i, respectively, (6) leads to

$$ \boldsymbol{e}_{v}(t)=-k_{c}(\boldsymbol{L}\otimes \boldsymbol{I}_{p}){\int}_{t_{0}}^{t}\boldsymbol{e}_{v} d\tau-\tilde{\boldsymbol{x}}_{v} $$
(54)

in which, from (40), \(\tilde {\boldsymbol {x}}_{v}\) converges to the origin in finite-time. Finally, since the communication graph is connected, the immediate consequence of (54) is that \({\int \limits }_{t_{0}}^{t}\boldsymbol {e}_{v,i}={\int \limits }_{t_{0}}^{t}\boldsymbol {e}_{v,j}\), ∀i, j which, based on (42) and (52), implies that ev, i = 0pi. This completes the proof.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lippi, M., Marino, A. Human Multi-Robot Physical Interaction: a Distributed Framework. J Intell Robot Syst 101, 35 (2021). https://doi.org/10.1007/s10846-020-01277-y

Download citation

Keywords

  • Human-robot interaction
  • Shared control
  • Distributed control