Combining Semi-autonomous Navigation with Manned Behaviour in a Cooperative Driving System for Mobile Robotic Telepresence

  • Andrey KiselevEmail author
  • Annica Kristoffersson
  • Amy Loutfi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8928)


This paper presents an image-based cooperative driving system for telepresence robot, which allows safe operation in indoor environments and is meant to minimize the burden on novice users operating the robot. The paper focuses on one emerging telepresence robot, namely, mobile remote presence systems for social interaction. Such systems brings new opportunities for applications in healthcare and elderly care by allowing caregivers to communicate with patients and elderly from remote locations. However, using such systems can be a difficult task particularly for caregivers without proper training. The paper presents a first implementation of a vision-based cooperative driving enhancement to a telepresence robot. A preliminary evaluation in the laboratory environment is presented.


Human-robot interaction Mobile robotic telepresence Teleoperation User interfaces 


  1. 1.
    Beer, J.M., Takayama, L.: Mobile remote presence systems for older adults: acceptance, benefits, and concerns. In: Proceedings of the 6th International Conference on Human-Robot Interaction HRI 11. HRI 2011, pp. 19–26. ACM (2011)Google Scholar
  2. 2.
    Kristoffersson, A., Coradeschi, S., Loutfi, A.: A Review of Mobile Robotic Telepresence. Advances in Human-Computer Interaction 2013, 1–17 (2013)CrossRefGoogle Scholar
  3. 3.
    Gandsas, A., Parekh, M., Bleech, M.M., Tong, D.A.: Robotic telepresence: profit analysis in reducing length of stay after laparoscopic gastric bypass. Journal of the American College of Surgeons 205(1), 72–77 (2007)CrossRefGoogle Scholar
  4. 4.
    Coradeschi, S., Kristoffersson, A., Loutfi, A., Von Rump, S., Cesta, A., Cortellessa, G., Gonzalez, J.: Towards a methodology for longitudinal evaluation of social robotic telepresence for elderly. In: Proceedings of the HRI 2011 Workshop on Social Robotic Telepresence, pp. 1–7 (2011)Google Scholar
  5. 5.
    Kiselev, A., Loutfi, A.: Using a Mental Workload Index as a Measure of Usability of a User Interface for Social Robotic Telepresence. Workshop in Social Robotics Telepresence (2012)Google Scholar
  6. 6.
    Lee, M.K., Takayama, L.: Now, i have a body. In: Proceedings of the 2011 annual conference on Human factors in computing systems - CHI 2011, p. 33. ACM Press, New York (2011)Google Scholar
  7. 7.
    Engström, J., Johansson, E., Östlund, J.: Effects of visual and cognitive load in real and simulated motorway driving. Transportation Research Part F: Traffic Psychology and Behaviour 8(2), 97–120 (2005)CrossRefGoogle Scholar
  8. 8.
    Hankins, T.C., Wilson, G.F.: A comparison of heart rate, eye activity, EEG and subjective measures of pilot mental workload during flight. Aviation space and environmental medicine 69(4), 360–367 (1998)Google Scholar
  9. 9.
    Parasuraman, R., Riley, V.: Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors: The Journal of the Human Factors and Ergonomics Society 39(2), 230–253 (1997)CrossRefGoogle Scholar
  10. 10.
    Patten, C.J.D., Kircher, A., Ostlund, J., Nilsson, L., Svenson, O.: Driver experience and cognitive workload in different traffic environments. Accident Analysis & Prevention 38(5), 887–894 (2006)CrossRefGoogle Scholar
  11. 11.
    Vlassenroot, S., Broekx, S., Mol, J.D., Panis, L.I., Brijs, T., Wets, G.: Driving with intelligent speed adaptation: Final results of the Belgian ISA-trial. Transportation Research Part A: Policy and Practice 41(3), 267–279 (2007)Google Scholar
  12. 12.
    Urmson, C., Anhalt, J., Bagnell, D., Baker, C., Bittner, R., Clark, M.N., Dolan, J., Duggins, D., Galatali, T., Geyer, C., Gittleman, M., Harbaugh, S., Hebert, M., Howard, T.M., Kolski, S., Kelly, A., Likhachev, M., McNaughton, M., Miller, N., Peterson, K., Pilnick, B., Rajkumar, R., Rybski, P., Salesky, B., Seo, Y.W., Singh, S., Snider, J., Stentz, A., Whittaker, W.R., Wolkowicki, Z., Ziglar, J., Bae, H., Brown, T., Demitrish, D., Litkouhi, B., Nickolaou, J., Sadekar, V., Zhang, W., Struble, J., Taylor, M., Darms, M., Ferguson, D.: Autonomous driving in urban environments: Boss and the Urban Challenge. Journal of Field Robotics 25(8), 425–466 (2008)CrossRefGoogle Scholar
  13. 13.
    Bertozzi, M., Broggi, A.: GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection. IEEE transactions on image processing : a publication of the IEEE Signal Processing Society 7(1), 62–81 (1998)CrossRefGoogle Scholar
  14. 14.
    Labayrade, R., Royere, C., Gruyer, D., Aubert, D.: Cooperative Fusion for Multi-Obstacles Detection With Use of Stereovision and Laser Scanner. Autonomous Robots 19(2), 117–140 (2005)CrossRefGoogle Scholar
  15. 15.
    Aufrère, R., Gowdy, J., Mertz, C., Thorpe, C., Wang, C.C., Yata, T.: Perception for collision avoidance and autonomous driving. Mechatronics 13(10), 1149–1161 (2003)CrossRefGoogle Scholar
  16. 16.
    Maier, D., Stachniss, C., Bennewitz, M.: Vision-Based Humanoid Navigation Using Self-Supervized Obstacle Detection. International Journal of Humanoid Robotics 10(02), 1350016 (2013)CrossRefGoogle Scholar
  17. 17.
    Itoh, M., Horikome, T., Inagaki, T.: Effectiveness and driver acceptance of a semi-autonomous forward obstacle collision avoidance system. Proceedings of the Human Factors and Ergonomics Society 3, 2091–2095 (2010)CrossRefGoogle Scholar
  18. 18.
    Wegner, R., Anderson, J.: Balancing robotic teleoperation and autonomy for urban search and rescue environments. In: Tawfik, A.Y., Goodwin, S.D. (eds.) Canadian AI 2004. LNCS (LNAI), vol. 3060, pp. 16–30. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  19. 19.
    Doroodgar, B., Ficocelli, M., Mobedi, B., Nejat, G.: The search for survivors: Cooperative human-robot interaction in search and rescue environments using semi-autonomous robots. In: 2010 IEEE International Conference on Robotics and Automation, pp. 2858–2863. IEEE (May 2010)Google Scholar
  20. 20.
    Anderson, S.J., Karumanchi, S.B., Iagnemma, K., Walker, J.M.: The intelligent copilot: A constraint-based approach to shared-adaptive control of ground vehicles. IEEE Intelligent Transportation Systems Magazine 5(2), 45–54 (2013)CrossRefGoogle Scholar
  21. 21.
    Law, C.K.H., Xu, Y.: Shared control for navigation and balance of a dynamically stable robot. Proceedings - IEEE International Conference on Robotics and Automation 2, 1985–1990 (2002)Google Scholar
  22. 22.
    Yokokohji, Y., Ogawa, A., Hasunuma, H., Yoshikawa, T.: Operation modes for cooperating with autonomous functions in intelligent teleoperation systems. In: Proceedings - IEEE International Conference on Robotics and Automation, vol. 3, pp. 510–515. IEEE (1993)Google Scholar
  23. 23.
    Kiselev, A., Mosiello, G., Kristoffersson, A., Loutfi, A.: Semi-autonomous Cooperative Driving for Mobile Robotic Telepresence Systems. In: Proceedings of the 2014 ACM/IEEE International Conference on Human-robot Interaction. HRI 2014, 104. ACM New York (2014)Google Scholar
  24. 24.
    International Organization for Standartization: ISO 13482:2014 Robots and robotic devices - Safety requirements for personal care robots (2014)Google Scholar
  25. 25.
    Giraff Technologies AB: Giraff Technologies AB (2013)Google Scholar
  26. 26.
    Joblove, G.H., Greenberg, D.: Color Spaces for Computer Graphics. SIGGRAPH Comput. Graph. 12(3), 20–25 (1978)CrossRefGoogle Scholar
  27. 27.
    Stockman, G., Shapiro, L.G.: Computer Vision, 1st edn. Prentice Hall PTR, Upper Saddle River (2001) Google Scholar
  28. 28.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)Google Scholar
  29. 29.
    Bytedeco: JavaCV (2014)Google Scholar
  30. 30.
    Mosiello, G., Kiselev, A., Loutfi, A.: Using Augmented Reality to Improve Usability of the User Interface for Driving a Telepresence Robot. Paladyn, Journal of Behavioral Robotics 4(3), 174–181 (2013)Google Scholar
  31. 31.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)CrossRefGoogle Scholar
  32. 32.
    Fogel, I., Sagi, D.: Gabor filters as texture discriminator (1989)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Andrey Kiselev
    • 1
    Email author
  • Annica Kristoffersson
    • 1
  • Amy Loutfi
    • 1
  1. 1.Örebro UniversityörebroSweden

Personalised recommendations