Abstract
Situational awareness in remote environments is crucial for human operators to teleoperate mobile manipulators confidently and reliably. Visual feedback is the most common way for environment perception, providing rich information to human operators. This paper proposes an intuitive teleoperation method by combining gaze tracking and hand motion detection to teleoperate a mobile manipulator. A camera is fixed on the end-effector of the mobile robot’s arm to provide visual feedback, acting as the eye of the teleoperator. Rather than direct remote control of the robot, an adaptive auto-correction mechanism is introduced for helping human operators to achieve better hand-eye coordination of the teleoperation experiences. The mobile manipulator can adjust its behaviours, such as speed, while gaze and hand movements of the operator are in different states. The experiments carried out demonstrated the effectiveness of the proposed algorithm and the survey evaluation verified the practical application value of the system.
This work is supported by the Advanced Sustainable Manufacturing Technologies (ASTUTE) 2020 and ASTUTE East projects. We would like to thank the visiting MSc students from INSA-Strasbourg, France, for their previous work on the Smart Eye system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bischoff, R., Huggenberger, U., Prassler, E.: KUKA youBot-a mobile manipulator for research and education. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–4. IEEE (2011)
Brooke, J., et al.: SUS-a quick and dirty usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, I.L. (eds.) Usability Evaluation in Industry, pp. 4–7. CRC Press, London (1996)
Du, G., Zhang, P., Liu, X.: Markerless human-manipulator interface using leap motion with interval kalman filter and improved particle filter. IEEE Trans. Industr. Inf. 12(2), 694–704 (2016). https://doi.org/10.1109/TII.2016.2526674
Dziemian, S., Abbott, W.W., Faisal, A.A.: Gaze-based teleprosthetic enables intuitive continuous control of complex robot arm use: writing amp; drawing. In: 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), pp. 1277–1282, June 2016. https://doi.org/10.1109/BIOROB.2016.7523807
Ggo, D., Carreto, C., Figueiredo, L.: Teleoperation of a mobile robot based on eye-gaze tracking. In: 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–6, June 2017. https://doi.org/10.23919/CISTI.2017.7975673
Li, X., et al.: Active cannula robot with misorientation auto-recovery camera: a method to improve hand-eye coordination in minimally invasive surgery. In: 2013 13th International Conference on Control, Automation and Systems (ICCAS 2013), pp. 276–280, Oct 2013. https://doi.org/10.1109/ICCAS.2013.6703908
Peppoloni, L., Brizzi, F., Avizzano, C.A., Ruffaldi, E.: Immersive ROS-integrated framework for robot teleoperation. In: 2015 IEEE Symposium on 3D User Interfaces (3DUI), pp. 177–178, March 2015. https://doi.org/10.1109/3DUI.2015.7131758
Qiu, R., et al.: Towards robust personal assistant robots: experience gained in the SRS project. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1651–1657, Oct 2012. https://doi.org/10.1109/IROS.2012.6385727
Tunstel, E., Maimone, M., Trebi-Ollennu, A., Yen, J., Petras, R., Willson, R.: Mars exploration rover mobility and robotic arm operational performance. In: 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 1807–1814. IEEE (2005)
Xiang, Z., Su, J., Ma, Z.: Uncalibrated hand-eye coordination based HRI on humanoid robot. In: Proceedings of the 33rd Chinese Control Conference, pp. 8438–8443, July 2014. https://doi.org/10.1109/ChiCC.2014.6896416
Yu, M., Wang, X., Lin, Y., Bai, X.: Gaze tracking system for teleoperation. In: The 26th Chinese Control and Decision Conference (2014 CCDC), pp. 4617–4622, May 2014. https://doi.org/10.1109/CCDC.2014.6852997
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, J., Ji, Z., Niu, H., Setchi, R., Yang, C. (2019). An Auto-Correction Teleoperation Method for a Mobile Manipulator Using Gaze Tracking and Hand Motion Detection. In: Althoefer, K., Konstantinova, J., Zhang, K. (eds) Towards Autonomous Robotic Systems. TAROS 2019. Lecture Notes in Computer Science(), vol 11650. Springer, Cham. https://doi.org/10.1007/978-3-030-25332-5_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-25332-5_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25331-8
Online ISBN: 978-3-030-25332-5
eBook Packages: Computer ScienceComputer Science (R0)