Abstract
In this paper, a novel interface of human-robot interaction has been developed to provide enhanced user experience for teleoperators. The interface has been implemented and tested on a Baxter robot platform and it can be easily adapted to other robot platforms. The main objective of this work is to provide a teleoperator immersive experience when controlling a telerobot arm by enabling the user to see and feel what the robot sees and feels from a first person point of view. This objective has been achieved by our designed interface integrating a haptic feedback device, a virtual reality headset, and an RGB-D camera. An operator can manipulate a robotic arm and receive force feedback information about interactions between the robot’s grippers, as well as the robot’s environment, whilst viewing the captured visual information of the robot’s workspace, on the screen of the virtual reality headset. A servo motor driving platform has been designed as a new robot head to manipulate the camera on top of it, such that a teleoperator is able to control the pose of the camera in a natural manner via the wearable virtual reality headset. The orientation of the built-in inertial measurement unit (IMU) of the virtual reality headset is used to directly command the angles of the head platform on which the camera is mounted. The operator will have an immersive and in-depth experience when manipulating the robotic arm. Extensive tests with a variety of users have been carried out to evaluate the design in this work with quantified analysis.
This work was supported in part by EPSRC grants EP/L026856/1 and EP/J004561/1 (BABEL) as well as Royal Society Newton Mobility Grant IE150858.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Billard, A., Calinon, S., Dillmann, R., Schaal, S.: Robot programming by demonstration. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 1371–1394. Springer, Heidelberg (2008)
Sarmah, A., Gulhane, U.: Surgical robot teleoperated laparoscopic grasper with haptics feedback system. In: 2010 International Conference on Emerging Trends in Robotics and Communication Technologies, pp. 288–291, December 2010
Livatino, S., Muscato, G., Privitera, F.: Stereo viewing and virtual reality technologies in mobile robot teleguide. IEEE Trans. Robot. 25, 1343–1355 (2009)
Kofman, J., Wu, X., Luu, T.J., Verma, S.: Teleoperation of a robot manipulator using a vision-based human-robot interface. IEEE Trans. Industr. Electron. 52, 1206–1219 (2005)
Ju, Z., Yang, C., Li, Z., Cheng, L., Ma, H.: Teleoperation of humanoid Baxter robot using haptic feedback. In: 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, pp. 1–6, September 2014
Reddivari, H., Yang, C., Ju, Z., Liang, P., Li, Z., Xu, B.: Teleoperation control of Baxter robot using body motion tracking. In: 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, pp. 1–6, September 2014
Wang, X., Yang, C., Ma, H.: Automatic obstacle avoidance using redundancy for shared controlled telerobot manipulator. In: The 5th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent Systems, pp. 1338–1343, 8–12 June 2015
Liang, P., Yang, C., Li, Z., Li, R.: Writing skills transfer from human to robot using stiffness extracted from sEMG. In: The 5th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent Systems, pp. 19–24, 8–12 June 2015
Hou, X., Mahony, R.: Dynamic kinesthetic boundary for haptic teleoperation of vtol aerial robots in complex environments. IEEE Trans. Syst. Man Cybern. Syst. 46, 694–705 (2016)
Cho, S.K., Jin, H.Z., Lee, J.M., Yao, B.: Teleoperation of a mobile robot using a force-reflection joystick with sensing mechanism of rotating magnetic field. IEEE/ASME Trans. Mechatron. 15, 17–26 (2010)
Romano, J., Hsiao, K., Niemeyer, G., Chitta, S., Kuchenbecker, K.: Human-inspired robotic grasp control with tactile sensing. IEEE Trans. Robot. 27, 1–10 (2011)
Lee, S., Sukhatme, G., Kim, G.J., Park, C.M.: Haptic teleoperation of a mobile robot: a user study. Presence 14, 345–365 (2005)
Tachi, S., Komoriya, K., Sawada, K., Nishiyama, T., Itoko, T., Kobayashi, M., Inoue, K.: Telexistence cockpit for humanoid robot control. Adv. Robot. 17(3), 199–217 (2003)
Kim, D.Y., Lee, M.S., Choi, S.H., Koo, K.-J., Hwang, I., Kim, Y.J.: An immersive telepresence platform based on distributed architecture. In: 2013 International Conference on ICT Convergence, pp. 465–467, October 2013
James, C.A., Bednarz, T.P., Haustein, K., Alem, L., Caris, C., Castleden, A.: Tele-operation of a mobile mining robot using a panoramic display: an exploration of operators sense of presence. In: 2011 IEEE International Conference on Automation Science and Engineering, pp. 279–284, August 2011
Conn, M.A., Sharma, S.: Immersive telerobotics using the oculus rift and the 5DT ultra data glove. In: 2016 International Conference on Collaboration Technologies and Systems, pp. 387–391, October 2016
Regenbrecht, J., Tavakkoli, A., Loffredo, D.: A robust and intuitive 3d interface for teleoperation of autonomous robotic agents through immersive virtual reality environments. In: 2017 IEEE Symposium on 3D User Interfaces, pp. 199–200, March 2017
Dünser, A., Lochner, M., Engelke, U., Fernández, D.R.: Visual and manual control for human-robot teleoperation. IEEE Comput. Graph. Appl. 35, 22–32 (2015)
Chen, J., Glover, M., Li, C., Yang, C.: Development of a user experience enhanced teleoperation approach. In: 2016 International Conference on Advanced Robotics and Mechatronics, pp. 171–177, August 2016
Rethink Robotics, May 2014. http://sdk.rethinkrobotics.com/mediawiki-1.22.2/images/thumb/4/48/Hand_sensors.png/425px-Hand_sensors.png. Accessed 01 July 2015
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendix
Participant Questionnaire
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Chen, J., Glover, M., Yang, C., Li, C., Li, Z., Cangelosi, A. (2017). Development of an Immersive Interface for Robot Teleoperation. In: Gao, Y., Fallah, S., Jin, Y., Lekakou, C. (eds) Towards Autonomous Robotic Systems. TAROS 2017. Lecture Notes in Computer Science(), vol 10454. Springer, Cham. https://doi.org/10.1007/978-3-319-64107-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-64107-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-64106-5
Online ISBN: 978-3-319-64107-2
eBook Packages: Computer ScienceComputer Science (R0)