Skip to main content
SpringerLink
Log in

Development of a Shared Controller for Obstacle Avoidance in a Teleoperation System

  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

Abstract

Several methods have been investigated to increase the efficiency of the operator in teleoperation, but remote devices still cannot be operated efficiently in the presence of the obstacle. In this study, a virtual link and virtual joints were created within the end-effector of the slave robot, and a shared controller was designed to implement an effective obstacle avoidance algorithm for the remote control system. Teleoperation experiments were conducted to verify the algorithm. Completion time and the NASA Task Load Index (NASA-TLX) were measured to evaluate the improvement of teleoperator work efficiency. When the obstacle avoidance algorithm was used, completion time decreased by 8.64%, and the average NASA-TLX decreased by 30.33 % as compared without the algorithm. Our method effectively improved completion time and NASA-TLX scores for both skilled and nonskilled human-operators.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. X. Tang, D. Zhao, H. Yamada, and T. Ni, “Haptic interaction in teleoperation control system of construction robot based on virtual reality,” Proc. of International Conference on Mechatronics and Automation, IEEE, pp. 78–83, 2009.

  2. V. M. Hung and U. J. Na, “Tele-operation of a 6-dof serial robot using a new 6-dof haptic interface,” Proc. of IEEE International Symposium on Haptic Audio Visual Environments and Games, pp. 1–6, 2010.

  3. T. Chen, D. Zhao, and Z. Zhang, “Research on the teleoperation robot system with tele-presence based on the virtual reality,” Proc. of IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 302–306, 2007.

  4. O. Khatib, “Real-time obstacle avoidance for manipulators and mobile robots,” The international journal of robotics research, vol. 5, no. 1, pp. 90–98, 1986.

    Article  Google Scholar 

  5. A. Muller, “Collision avoiding continuation method for the inverse kinematics of redundant manipulators,” Proceedings of the IEEE International Conference on Robotics and Automation, vol. 2, pp. 1593–1598, 2004.

  6. H. Reimann, I. Iossifidis, and G. Schoner, “Generating collision free reaching movements for redundant manipulators using dynamical systems,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5372–5379, 2010.

  7. J. Kim, D. Pae, and M. Lim. “Obstacle avoidance path planning based on output constrained model predictive control,” International Journal of Control, Automation and Systems, vol. 17, no. 11, pp. 2850–2861, 2019.

    Article  Google Scholar 

  8. R. Quirynen, K. Berntorp, and S. di Cairano, “Embedded optimization algorithms for steering in autonomous vehicles based on nonlinear model predictive control,” Proceedings of the IEEE Annual American Control Conference (ACC), pp. 3251–3256, 2018.

  9. F. Borrelli, P. Falcone, T. Keviczky, J. Asgari, and D. Hrovat, “MPC-based approach to active steering for autonomous vehicle systems,” International Journal of Vehicle Autonomous System, vol. 3, no. 2–4, pp. 265–291, 2005.

    Article  Google Scholar 

  10. Y. Zheng, S. E. Li, K. Li, F. Borrelli, and J. K. Hedrick, “Distributed model predictive control for heterogeneous vehicle platoons under unidirectional topologies,” IEEE Transaction on Control Systems Technology, vol. 25, no. 3, pp. 899–910, 2017.

    Article  Google Scholar 

  11. J. J. Abbott, P. Marayong, and A. M. Okamura, “Haptic virtual fixtures for robot-assisted manipulation,” Robotics Research, Springer, pp. 49–64, 2007.

  12. M. Wrock and S. Nokleby, “Haptic teleoperation of a manipulator using virtual fixtures and hybrid positionvelocity control,” IFToMM World Congress in Mechanism and Machine Science, 2011.

  13. F. Ferraguti, N. Preda, M. Bonfe, and C. Secchi, “Bilateral teleoperation of a dual arms surgical robot with passive virtual fixtures generation,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4223–4228, 2015.

  14. D. Aarno, S. Ekvall, and D. Kragic, “Adaptive virtual fixtures for machine-assisted teleoperation tasks,” Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1139–1144, 2005.

  15. A. B. Kuang, S. Payandeh, B. Zheng, F. Henigman, and C. L. MacKenzie, “Assembling virtual fixtures for guidance in training environments,” Proc. of 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 367–374, 2004.

  16. S. Payandeh and Z. Stanisic, “On application of virtual fixtures as an aid for telemanipulation and training,” Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 18–23, 2002.

  17. C. W. Wampler, “Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 16, no. 1, pp. 93–101, 1986.

    Article  Google Scholar 

  18. S. I. An and D. Lee, “Prioritized inverse kinematics with multiple task definitions,” Proc. of IEEE International Conference on Robotics and Automation, pp. 1423–1430, 2015.

  19. J. Xiang, C. Zhong, and W. Wei, “General-weighted least-norm control for redundant manipulators,” IEEE Transactions on Robotics, vol. 26, no. 4, pp. 660–669, 2010.

    Article  Google Scholar 

  20. F. Flacco, A. de Luca, and O. Khatib, “Motion control of redundant robots under joint constraints: Saturation in the null space,” Proc. of IEEE International Conference on Robotics and Automation, pp. 285–292, 2012.

  21. L. Zlajpah and B. Nemec, “Kinematic control algorithms for on-line obstacle avoidance for redundant manipulators,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2, pp. 1898–1903, 2002.

    Article  Google Scholar 

  22. K. Glass, R. Colbaugh, D. Lim, and H. Seraji, “Real-time collision avoidance for redundant manipulators,” IEEE Transactions on Robotics and Automation, vol. 11, no. 3, pp. 448–457, 1995.

    Article  Google Scholar 

  23. X. Wang, C. Yang, H. Ma, and L. Cheng, “Shared control for teleoperation enhanced by autonomous obstacle avoidance of robot manipulator,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4575–4580, 2015.

  24. S. Luo, H. Lu, J. Xiao, Q. Yu, and Z. Zheng, “Robot detection and localization based on deep learning,” Proc. of Chinese Automation Congress (CAC), pp. 7091–7095, 2017.

  25. J. Han, K. Cho, H. Choi, and G.-H. Yang, “Detected obstacle avoidance in teleoperation system with a virtual link and virtual joints,” Proc. of 15th International Conference on Ubiquitous Robots (UR), pp. 108–111, 2018.

  26. J. Redmon and A. Farhadi, “Yolo9000: better, faster, stronger,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271, 2016.

  27. J. A. Hartigan and M. A. Wong, “Algorithm as 136: A kmeans clustering algorithm,” Journal of the Royal Statistical Society, Series C (Applied Statistics), vol. 28, no. 1, pp. 100–108, 1979.

    Google Scholar 

  28. S. G. Hart and L. E. Staveland, “Development of nasatlx (task load index): Results of empirical and theoretical research,” Advances in Psychology, Elsevier, vol. 52, pp. 139–183, 1988.

  29. J. G. Wildenbeest, D. A. Abbink, C. J. Heemskerk, F. C. Van Der Helm, and H. Boessenkool, “The impact of haptic feedback quality on the performance of teleoperated assembly tasks,” IEEE Transactions on Haptics, vol. 6, no. 2, pp. 242–252, 2013.

    Article  Google Scholar 

  30. J. Yang, M. Kamezaki, R. Sato, H. Iwata, and S. Sugano, “Inducement of visual attention using augmented reality for multi-display systems in advanced tele-operation,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5364–5369, 2015.

  31. A. Hacinecipoglu, E. I. Konukseven, and A. B. Koku, “Evaluation of haptic feedback cues on vehicle teleoperation performance in an obstacle avoidance scenario,” Proc. of World Haptics Conference (WHC), pp. 689–694, 2013.

  32. A. Kanso, I. H. Elhajj, E. Shammas, and D. Asmar, “Enhanced teleoperation of UAVs with haptic feedback,” Proc. of IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pp. 305–310, 2015.

  33. D. Powell and M. K. O’Malley, “The task-dependent efficacy of shared-control haptic guidance paradigms,” IEEE Transactions on Haptics, vol. 5, no. 3, pp. 208–219, 2012.

    Article  Google Scholar 

  34. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and F.-F. Li, “Imagenet large scale visual recognition challenge,” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Robotics R&D Group, Korea Institute of Industrial Technology, Ansan, 15588, Korea

    JiWoong Han, Kyunghwan Cho, Inhoon Jang & Gi-Hun Yang

  2. School of Robotics and Virtual Engineering, University of Science & Technology, Daejon, 34113, Korea

    JiWoong Han & Gi-Hun Yang

  3. Department of Rural and Biosystems Engineering, Chonnam National University, Gwangju, 61186, Korea

    Chanyoung Ju & Hyoung Il Son

Authors
  1. JiWoong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kyunghwan Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Inhoon Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chanyoung Ju
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hyoung Il Son
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gi-Hun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gi-Hun Yang.

Additional information

Recommended by Associate Editor Changchun Hua under the direction of Editor Myo Taeg Lim.

This work was supported by the Technology Innovation Program (Industrial Strategic Technology Development Program, 10060070, Development of Core Teleoperation Technologies for Maintaining and Repairing Tasks in Nuclear Power Plants) funded by the Ministry of Trade, Industry and Energy (MOTIE, Korea).

JiWoong Han received his B.S. degree in Mechatronics Engineering from Chungnam National University in 2014. Since 2015, he has been a Ph.D. student in the School of Robotics and Virtual Engineering, University of Science & Technology. His research interests include teleopeation, shared control, and machine learning.

Kyunghwan Cho received his B.S. degree in School of electrical and electronics engineering from Soongsil University and his M.S degree in electronic and electrical engineering from SungKyunKwan University, in 2014 and 2016, respectively. He was a researcher with the Robotics R&D Group, Korea Institute of Industrial Technology from 2016 to 2019. His research interests include deep learning, object detection.

Inhoon Jang received his B.S., M.S. and Ph.D. degrees from the Department of Electrical and Electronics Engineering, Chung-Ang University, Seoul, Korea, in 1993, 1999 and 2010, respectively. He is currently a principal researcher in the Korea Institute of Industrial Technology. His research interests include Robotic perception & control in dynamic, unstructured environments.

Chanyoung Ju received his B.S. and M.S. degrees from the Department of Rural and Biosystems Engineering, Chonnam National University, Korea, in 2017 and 2019, respectively, where he is currently pursuing a Ph.D. degree in biosystems engineering. His research interests include field robotics, supervisory control, and discrete event and hybrid systems.

Hyoung Il Son received his B.S. and M.S. degrees from the Department of Mechanical Engineering, Pusan National University, Korea, in 1998 and 2000, respectively, and a Ph.D. degree from the Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Korea, in 2010. In 2015, he joined the Faculty of the Department of Rural and Biosystems Engineering, Chonnam National University, Gwangju, Korea, where he is currently an Associate Professor. Before joining Chonnam National University, from 2012 to 2015, he led the Telerobotics Group, Central Research Institute, Samsung Heavy Industries, Daejeon, Korea, as a Principal Researcher. He also had several appointments both academia and industry, as a Senior Researcher, with LG Electronics, Pyungtaek, Korea, from 2003 to 2005, and Samsung Electronics, Cheonan, Korea, from 2005 to 2009, a Research Associate with the Institute of Industrial Science, The University of Tokyo, Tokyo, Japan, in 2010, and a Research Scientist with the Max Planck Institute for Biological Cybernetics, Tübingen, Germany, from 2010 to 2012. His research interests include field robotics, hybrid systems, teleoperation, and haptics.

Gi-Hun Yang received his B.S. degree in Mechanical Engineering from the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 2000, an M.S. degree in Mechanical Engineering from KAIST, Daejeon, Korea, in 2002, and a Ph.D. degree in Mechanical Engineering from KAIST, in 2008. He was previously a Post-Doc at the Center for Cognitive Robotics Research at KIST. He is currently a Principal Research Scientist of Robotics Group at Korea Institute of Industrial Technology (KITECH). His current research interests include haptics, teleoperation, HRI, bio-inspired robots and industrial robots.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, J., Cho, K., Jang, I. et al. Development of a Shared Controller for Obstacle Avoidance in a Teleoperation System. Int. J. Control Autom. Syst. 18, 2974–2982 (2020). https://doi.org/10.1007/s12555-019-0410-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-019-0410-0

Keywords

Search

Navigation